Twitter has released the code that determines which tweets appear on your timeline to GitHub and has post a blog post explaining the decision. It analyzes what the algorithm looks at when deciding which tweets to show in the For You timeline and how it ranks and filters them.
According to Twitter’s blog post, the recommendation pipeline consists of three main stages. First, it collects “the best tweets from various recommendation sources,” then ranks those tweets using “a machine learning model.” It then filters out tweets from people you’ve blocked, tweets you’ve already seen, or tweets that aren’t safe for work.
The post also takes a closer look at each step of the process. For example, it notes that the first step looks at about 1,500 tweets, and that the goal is to make the For You timeline around 50 percent tweets from people you follow (which are called “In-Network”), and 50 percent tweets from off-network accounts you don’t follow. It also says that the ranking is meant to “optimize for positive engagement (e.g. likes, retweets, and replies)”, and that the last step will try to make sure you don’t see too many tweets from the same person .
CEO Elon Musk has been promising the move for some time — on March 24, 2022, before owning the site, he polled his followers about whether Twitter’s algorithm should be open-source, and about 83 percent of responses said “Yes”. In February, he promised it would happen within a week, before that postpone the deadline until March 31 earlier this month.
Musk tweeted that Friday’s release was “Most of the Recommendation Algorithm”, and said the rest would be released in the future. He also said the hope is “that independent third parties can determine with reasonable accuracy what is likely to be shown to users.”
Musk has been prepares his audience for disappointment in the algorithm when they see it (which of course is a big assumption that people will actually understand the complex code). He said it is “overly complicated and not fully understood internally”, and that people will “discover a lot of silly things”, but has promised to solve problems as soon as they are discovered. “Providing code transparency will be incredibly embarrassing at first, but it should lead to a rapid improvement in recommendation quality,” he tweeted.
There’s a difference between code transparency, where users can see the mechanisms that choose tweets for their timelines, and code that’s open-source, where the community can submit their own code for consideration and use the algorithm in other projects. While Musk has said it will be open source, Twitter will have to do the work if it wants to earn that label. That includes figuring out systems of governance that decide which pull requests to approve, which user-raised issues deserve attention, and how to stop bad actors from sabotaging the code for their own ends.
Twitter says people can submit pull requests that may eventually end up in the codebase
The company says it’s working on this – the readme for the GitHub says “we invite the community to submit GitHub issues and get requests for suggestions on how to improve the recommendation algorithm.” However, it goes on to say that Twitter is still building “tools to manage these suggestions and sync changes across our internal repository.”
The decision to increase transparency around its recommendations is not happening in a bubble. Musk has openly criticized the way Twitter’s previous management handled moderation and recommendation, orchestrating a barrage of stories that he claimed would expose the platform’s “suppression of freedom of expression”. (Mostly it just served to show how normal content moderation works.)
But now that he’s in charge, he’s also faced a lot of backlash — from users annoyed about their For You pages shoving his tweets in their faces, to his conservative pushers increasingly worried about how little engagement they’re getting. to get. He quarreled that negative and hateful content is “deboosted to the max” in the site’s new recommendation algorithms, a claim outside analysts without access to the code.
Twitter may also face some competition from the open source community. Mastodon, a decentralized social network, is gaining popularity in some circles, and Twitter co-founder Jack Dorsey is backing another similar project called Bluesky, which is built on an open-source protocol.
