For any social network, not just a federated one.
My thoughts: The way it works in big tech social networks is like this:
- **The organic methods: **
- your followee shares something from a poster you don’t follow
- someone you don’t follow comments on a post from someone you follow
- you join a group or community and find others you currently don’t follow
- The recommendation engine methods: content you do not follow shows up, and you are likely to engage in it based on statistical models. Big tech is pushing this more and more.
- Search: you specifically attempt to find what you’re looking for through some search capability. Big tech is pushing against this more and more.
In my opinion, the fediverse covers #1 well already. But #1 has a bubble effect. Your followees are less likely to share something very drastically different from what you already have.
The fediverse is principally opposed to #2, at least the way it is done in big tech. But maybe some variation of it could be done well.
#3 is a big weakness for fediverse. But I am curious how it would ideally manifest. Would it be full text search? Semantic search? Or something with more machine learning?
You must log in or register to comment.
For me, it’s full text search.
I tend to want to find an opinion on something very specific, so if I can just toss a phrase or model number or name of something into a search field and get actual non-AI, non-advertisement, non-stupid-shit results, that’d be absolutely ideal.
Like, say, how Google worked 15 years ago.
That was a big complaint during the 2022 migration. And it’s something that’s basically available on every fediverse platform not called Mastodon. I wish that fact had caused more people to actually check out those platforms, rather than further entrench Mastodon as the core of the fediverse.
The problem I ran into is that every single platform that primarily interacted with Mastodon (The keys, etc.) had the same exact same set of problems.
While yes, my Firefish instance had search, what was it searching? Local data only, and once I figured out that Mastodon-style replies didn’t federate to all of someone’s followers, it became pretty clear that it was uh, not very useful.
You can search, but any given server may or may not have access to data you actually want and thus, well, you just plain cannot meaningfully search for shit unless you go to one of the mega instances, or join giant piles of relays and store gigabyte upon gigabyte upon gigabyte of garbage data you do not care about.
The whole implementation is kinda garbage for search-based discovery from it’s very basic design all the way through to everyone’s implementations.
Search will never search non-local content. That’s not how search works anywhere. Even Google is searching local content.
That’s what search engine spiders do. They create local profiles of websites that end up being sorted and searched.
We have to give up on the idea of having easy and direct access to the whole of thw fediverse. Fefiverse sites don’t even know about every other fediverse site, and they never will.
Search will never search non-local content.
Which is the point I’m trying to make: right now, you cannot use search as a discoverability medium, unless you’re on something the scale of mastodon.social.
Search with a focus on new content discoverability is utterly useless for smaller or single user instances, because a search that only finds things you already know about isn’t exactly a useful search for discoverability.
If I have to be on the biggest instances, then there’s very little difference between something like Bluesky and Mastodon in terms of usability, and uh, I might as well pick the one that’s more likely to have the most growth and diversity of content.
We have to give up on the idea of having easy and direct access to the whole of thw fediverse.
I agree, and it’s why I’ve pretty much migrated back to centralized services with the exception of Lemmy, because Lemmy works very well in terms of finding useful shit to follow in a way that literally no other federated platform does.
It’s available? Where and how? Lemmy doesn’t seem to have a solid search, although it does have something.
I personally wouldn’t mind algorithmic recommendations if:
- you can control or choose the algorithm
- you can turn it off, or it turns off after you follow N amount of users
Discovery is important when you’re initially signing up, but once you found the people you want to follow, you don’t really need it any more. It should just be there to help new users, essentially. As long as it’s open source and not run for profit, there’s not the traditional incentive to keep your eyeballs on the app like we see with the other networks.
Search: you specifically attempt to find what you’re looking for through some search capability. Big tech is pushing against this more and more.
Funny, considering that you cannot find threads, communities, etc. that aren’t already federated with your instance because of someone being subscribed to it beforehand.
It’s trickier, but not impossible. Lemmyverse.net is a good starting point for community search
Yes, I was speaking about what would be ideal, and not what is possible today in the fediverse.
A search service could solve this issue.
If you want some search that covers as much of the Fediverse as possible, you’ll have to make it centralised in some way.
If you want some search that actually always covers all of the Fediverse all the way to instances that have only just been launched for the first time, you’ll have to make it fully centralised and hard-code that search into all Fediverse server apps. That way, when you first start your new private instance of whatever, it can immediately connect itself to that search engine and push any and all content on your instance to that search engine.
I don’t know how well you know the Fediverse outside Lemmy. But at least on Mastodon, but probably not only there, many of those who have been around for long enough would rebel against centralised search because they don’t want the Fediverse to rely on anything centralised.
Also, especially on Mastodon, you have those who strongly opposed the introduction of full-text search on Mastodon itself on the ground of full-text search being used in the Birdcage to find and then harass BIPoC and members of the 2SLGBTQIA+ community, many of whom have escaped to Mastodon because it did not have full-text search! There may actually still be Mastodon instances that run Mastodon 3.x in order to avoid the full-text search that was introduced with Mastodon 4.0.
Thus, covering exactly 100% of the Fediverse (the public Fediverse at least) would even be impossible with a centralised search engine. That is, unless that search engine managed to circumvent instance-wide blocks (you can be sure that places such as tech.lgbt or transfem.social would block the hell out of such a search engine) and ignore any and all kinds of search opt-outs.
I wish we had a nice tagging system (and I don’t think they should be hashtags) that was also in common use.
I want to be able to search any post related a certain topic, and sometimes, these may not always be in that topic’s community, because topics can overlap. For example, I might want to read posts about Ukraine war, but those might be in world news, US news, or combat footage communities. Could be a community about Ukraine in general, or Ukraine war specifically.
I also may not want to get it from a single Ukraine community. Maybe by finding posts with the “Ukraine war” tag, I’ll see several communities and join the one I want. But there needs to be a way to group them somehow.
Such a tag system may be useful for combined topics. For example, I may want to look for posts about music software. They might not be common in the music community, or software communities. But I could filter by both tags and find what I want.
Given the way things are in my perspective, what I want on mbin & lemmy is somewhere like a mix of 1 & 2, with 3 as a solid option. I know that the torches and pitchforks are about to come out, but I’ll try to outline the way I see it.
When I’m in a meme-scrolling mood, I have to look up meme magazines / communities to start (Method #3). Fine, that’s working as intended. Obviously that will lead naturally to Method #1; as I subscribe and gradually follow other posters, my bubble will grow.
But what I want for the ‘threadiverse’ is a more unified suggested page. If I’m in, let’s say, memes@lemmy.world, I’d like to also have my feed show content from memes@fedia.io, or lemm.ee, or whatever other threadiverse instances that my chosen instance is federating with. I’d also like to see “subject memes” on my meme feed as a default - Science Memes, Star Trek Memes, etc… That falls under Method #2 - because I want the software to predict that because I’ve subscribed to memes@*, and interacted with content from memes@*+1, that I will also like *memes*@*. Obviously this could also be a matter of tagging and magazine integration, but that’s something that would help the fediverse feel more united and less daunting for people.
Obviously dealing with the microblog side, mandatory tags or some form of community selection would be great to help out. It would be nice to see more microblog entries from Mastodon, Misskey, Pleroma, etc., sorted into magazine-like collections by tags.
If I’m in, let’s say, memes@lemmy.world, I’d like to also have my feed show content from memes@fedia.io, or lemm.ee, or whatever other threadiverse instances that my chosen instance is federating with.
When you say “feed” you mean your general news feed?
What if I only liked memes from memes@fedia.com, and other meme communities were too normie or boring for me? You’re going back to the issue with big tech social media, where they push on you what you didn’t sign up for, and you don’t necessarily like it!
I’m not against a recommendation engine, but it needs to be a lot more intentional from the user, and more transparent. I really dislike the “were just gonna push content you didn’t ask for here, but we think you’ll like it!”. No user choice, no transparency.
Btw, you should look into Quiblr. It’s a lemmy client that does sort of what you want. It has a built in recommendation engine, and it watches your engagement metrics to determine what you’ll like more of. The only thing it may not have is recommending you communities that aren’t visible to your instance (because no one on your instance follows it).
When I say “feed”, I mean the general homepage I see when I log into my account, rather than my Subscribed, Following, etc. views. I understand your concern, but if other related communities don’t suit you, then you’re free to block them as they come up. I think a ratio of 4 “in-bubble” to 1 “related” post would be fair. Maybe there could even be a slider somewhere depending on your software.
One of the few reasons I’ve never minded that part of the presentation on larger social media sites is that they operate on an opt-out model compared to the fediverse’s (current) opt-in model. But I think there’s enough transparency in “you like memes@fedia.io, here’s content from other groups we know with names containing ‘memes’”.
I may have to try out Quiblr, but I strongly prefer kbin/mbin to Lemmy, because I enjoy interacting with both the microblog side and the thread side of the fediverse on a single account. If mbin ever gets a video tab (for Loops & PeerTube), I’m going to jump for joy.
Looking back at my own life, I found the first few online communities I ever seriously joined (when I was a preteen, for context) through a web search, then discovered most others (recursively) from there, until I ended up (among other places) here on lemmy (which I can trace back to reddit, which I can trace back to a forum I started to pay attention to because of one of these original online communities preteen me found through a web search; not providing more info for privacy reasons). :P
So #1 and #3 are how it should work, IMHO, although #3 mostly for people who aren’t yet engaging with anything at all, most things will be discovered through #1.
I think most people use the Internet not for posting anything (or at least not much) themselves, but for looking up things they want to know (through a web search). In the pre-smartphone era, web searches would often direct to specific websites which might have forums attached to them, that was how I first started to seriously engage in my first online community actually. This isn’t the case much nowadays: many search results are either wikis (which are communities themselves, but don’t really invite discussion that isn’t about working on the wiki) or blogs/WordPress websites which may or may not have a comment section, but it’s relatively rare for them to have forums or even to link to reddit/fediverse communities to discuss their subject matter.
So I think it would be desirable if we managed to change that last part: top search results for many terms on search engines should be, or link to, fediverse communities, which should make it clear that users are invited to join. That would help us get more users engaged with fediverse communities in the first place, they would naturally discover more communities once they’re here.
A unified fediverse search service would be awesome, and its something I may try to tackle in the future. Part of why I’m asking this question here!
I have already found my instance’s “all communities” link fairly useful for finding communities.
The problem is I am subscribed to many communities that hardly anyone ever posts anything to, and the answer is not always “be the change you want to see in the world”. For example, I’m a native speaker of German and enjoy helping learners of German with grammatical questions, so I am subscribed to !german@lemmy.world – yet, almost no one ever posts any questions there for me to answer. (This is in stark contrast to reddit, where there is a very active /r/german.) People who see that community on lemmy probably think no one will ever read their question if they post it there. Chicken and egg problem.
I am a bit confused, and have a feeling you replied to the wrong comment somehow?
No, I was somewhat expanding on my previous thoughts on how to discover things on the fediverse and make it more active. Maybe that was a bit off-topic, sorry if it was.
Something I’ve thought about a bunch re: recommendation engines is the idea of a “sweet spot” that balances exploration and safety
Though actually I should start by saying that recommendation engines tend to aim to maximise engagement, which is why manosphere type content is so prevalent on places like YouTube if you go in with a fresh account — outrage generates engagement far more reliably than other content. I’m imagining a world where recommendation algorithms may be able to be individually tailored and trained, where I can let my goals shape the recommendations. I did some tinkering with a concept like this in the context of a personal music recommender, and I gave it an “exploration” slider, where at maximum, it’d suggest some really out-there stuff, but lower down might give me new songs from familiar artists. That project worked quite well, but it needs a lot of work to untangle before I can figure out how and why it worked so well.
That was a super individualistic program I made there, in that it was trained exclusively from data I gave it. One can get individual goals without having to rely on the data of just one person though - listenbrainz is very cool — its open source, and they are working on recommendation stuff (I’ve used listenbrainz as a user, but not yet as a contributor/developer)
Anyway, that exploration slider I mentioned is an aspect of the “sweet spot” I mentioned at the start. If we imagine a “benevolent” (aligned with the goals of its user) recommendation engine, and say that the goal you’re after is you want to listen to more diverse music. For a random set of songs that are new to you, we could estimate how close they are to your current taste (getting this stuff into matrices is a big chunk of the work, ime). But maybe one of the songs is 10 arbitrary units away from the boundary of your “musical comfort zone”. Maybe 10 units is too much too soon, too far away from your comfort zone. But maybe the song that’s only 1 unit away is too similar to what you like already and doesn’t feel stimulating and exciting in the way you expect the algorithm to feel. So maybe we could try what we think is a 4 or 5. Something novel enough to be exciting, but still feels safe.
Research has shown that recommendation algorithms can change affect our beliefs and our tastes [citation needed]. I got onto the music thing because I was thinking about the power in a recommendation algorithm, which is currently mostly used on keeping us consuming content like good cash cows. It’s reasonable that so many people have developed an aversion to algorithmic recommendations, but I wish I could have a dash of algorithmic exploration, but with me in control (but not quite so in control as what you describe in your options 3). As someone who is decently well versed in machine learning (by scientist standards — I have never worked properly in software development or ML), I think it’s definitely possible.
On art sites - i.e. DeviantArt and the various furry sites that copied it in the 2010s (many of which are still very much active) - every user has a “favorites” page to which they can add anyone else’s posts. This also sends a notification to the artist. When I get a favorite or a comment, I always check that artist’s posts and their favorites page, to see what else they have favorited, and I can reliably find new people and posts that way. I don’t think this is really analogous to a boost, since it doesn’t show up in your feed, but it’s also not quite the same as “likes”, since it’s much more expected that people will go looking through it (people don’t really look at other people’s likes much, even though they are almost always public). Plus it’s almost guaranteed that it’ll be filled with art, as opposed to blog or microblog style text posts, so you’re not as subjected to the hot takes of random people you don’t know.
It’s sort of like creating your own hand-curated feed for other people to see
(streams) and Forte have very nice user discovery: On your stream page which is always the landing page when you open your channel, there’s an area to the left which suggests two new contacts to you. The same thing is on your contacts page. Below the two suggestions, there is a button to a suggestions page which is basically a special, filtered version of the directory. There you get even more contact suggestions. The two suggestions are randomly picked from the two on your suggestions page.
If you fill out your profile, especially the keyword field, the suggestions will improve. Also, the suggestions may include not only users, but also whatever qualifies as a group, because the directory certainly does.
It’s basically like Facebook, but more convenient than Facebook has ever been AFAIR. Friendica has had this kind of suggestions since its inception in 2010, and (streams) is an indirect fork, so-to-speak. I don’t know if Friendica has a suggestion field up front nowadays, though; I know that Hubzilla doesn’t have one unless you add it using the PDL editor (and, please, which newbies dives into Hubzilla that deeply), and (streams) and Forte do have one.
There’s honestly nothing wrong with any of those options including 2. I get people see algorithm and recommendation and have aneurisms but this space isn’t looking to harm anyone intentionally. If there’s any space we should trust with any kind of algorithms it’s this space as there’s not the same incentives that Big Social has. As long as users can consent and have control.
But to answer your question it would take some hybrid search on top of an aggregator that is explicitly public and respects people’s flags. I know Mastodon is working on a discover service
