Elon's Twitter and Open Source Algorithm

By now, the entire world knows that Elon Musk owns Twitter. The ongoing frenzy surrounding this deal has reached peak “silly season” – a term political campaign operatives use to describe ridiculous and frivolous media stories near the end of a campaign season. (Coincidentally, we are in a real one now with the midterm elections looming next week.)

Reporters are camping out in front of Twitter’s San Francisco HQ to talk to laid off employees only to be hoaxed. Minute by minute coverage is being devoted to whether Elon will let Donald Trump back on. Seemingly serious people are spending time writing poems about Elon buying Twitter. (Seriously?!) Amidst all this noise, no one is talking about an important commitment that Elon made in this TED interview in April, shortly after he announced he wants to buy Twitter: open source Twitter’s algorithm.

Here's what Elon said:

“One of the things that I believe Twitter should do is open source the algorithm, and make any changes to people’s tweets, if they are emphasized or de-emphasized, that action should be made apparent so anyone can see that that action has been taken. So there’s no sort of behind the scenes manipulation, either algorithmically or manually.”

(Thunderous applause ensued)

Bringing transparency to social media algorithms is as important to the future of humanity (Elon’s stated life mission) as free speech, if not more. In fact, algorithmic transparency builds the trust necessary to strengthen free speech. The idea of open sourcing algorithms is not new. Many people have been advocating it for years. I’ve done the same in this newsletter regarding both TikTok and Facebook.

However, this idea has not been implemented widely, because it is often framed as a magnanimous decision that may be good for society, but is bad for business. This is a false tradeoff built on faulty logic. Open sourcing Twitter’s algorithm is actually good for humanity and good for business, whether it remains an ad business or becomes a subscription business. It is a rare case where you can have your cake and eat it too.

Let me explain.

Secret Algorithm is Not the Moneymaker

The algorithms that power social media platforms are often regarded as their “secret sauce”. This secrecy is misplaced, because the algorithms themselves are not where the business value lies.

How so?

Well, in the context of an ad-driven social media business, these algorithms are like maps, and their purpose is to:

• Route the right ads to the right users, so these users will do something that the advertisers who paid for the ads care about (e.g. click, “like”, buy something, etc).
• If the algorithmic route is successful, the advertisers are happy, Twitter generates revenue. Twitter can then try to charge a higher ad rate from these advertisers or other similar advertisers to make more money.
• If the algorithmic route fails, it adapts (sometimes with human intervention, sometimes automatically) by changing the weights of different parameters in the algorithm, so the “routes” are slightly different the next time another ad comes through. The algorithm then (hopefully) does a better job of routing, so Twitter can keep this advertiser’s business.

The algorithms are the “rules of the road” that govern all the routes. They are most likely built from existing AI frameworks, most of which are already open sourced (e.g. Tensorflow, PyTorch, Keras, etc.). Most of the mathematical equations that underpin these frameworks are published in academic journals, open to anyone who can understand them to implement, verify, and backtest. Twitter’s algorithms are likely built from or heavily leverages these free, open components. No one builds algorithms from scratch behind closed doors anymore thanks to the popularity of open source technologies. So there is nothing valuable about keeping the algorithm a “secret”.

What is valuable is the user-generated content (i.e. tweets) and the data that gets attached to that content (i.e. who viewed, liked, retweeted, replied, etc.).

“Liked” a Vitalik tweet about The Merge? Route an ad about a web3 conference to this user. User registered for the conference. Success! Cha-ching! Retweeted a tweet about Tesla by Cathie Wood? Route an ad about the new Prius paid by Toyota to that user. User ignores the ad (probably wants a pure EV, not a hybrid). Fail! Update route to send a different ad about a pure EV paid by Rivian. User clicked on it. Success! Cha-ching! You get the picture.

This data produces the signals that these algorithms need to do the routing and improve its routes over time, so it can get better at directing future ads in front of the right users.

An algorithm without data is like a highway system without cars – cool-looking but useless.

Good for Ads and Subscriptions

So far, Elon has given mixed signals about whether Twitter will continue to be an ad-driven business or switch gears to become a subscription-based business. Regardless of which model Twitter picks (most likely a mixture of both), open sourcing its algorithm is beneficial to both.

To sell more ads, it is business critical for Twitter to keep growing its user base, frequency of user-generated tweets, interactions with those tweets, and network effects between users, so the algorithms have more and better signals to improve its routes for the next ad. Having those algorithms be open sourced will only grow trust and transparency with users, and thus bring more users! Only a tiny percentage of users will go into the algorithm’s codebase to scrutinize the latest changes, but the fact that they can when they want to is what builds trust.

To pivot to subscriptions, Twitter has many success stories to draw from in the open source world. Although open source itself is not a business model, open source projects have been the foundation of many business models – open core, cloud-based SaaS, expert services – many of which are monetized via subscriptions. These models have powered many multi-billion dollar businesses, like Red Hat, MongoDB, Elastic, GitLab, Hashicorp, Databricks, to name a few. The open source projects and connected communities that these companies commercialize serve as a huge source of potential customers and a reservoir of trust (there is nothing more trust-building than trying something before you buy).

If Twitter open sources its algorithm and fosters a community alongside it, it can tap into the same “trust reservoir” to grow its subscription business. Instead of selling enterprise security features and SLAs, Twitter will sell blue check marks, edit features, and who knows what else (apparently for $8 dollars). (For more on open source, please see my previous writings on this topic over the last two years.)

There is no reason why Twitter cannot become the next big “open source” company. There is also no reason why Twitter cannot lead the way in algorithmic transparency and bring (or drag) Meta, TikTok, YouTube, and others along.

Twitter has been contributing to the open source ecosystem for many years, has a dedicated open source team, and maintains an active presence on GitHub. It already has the domain expertise to open source its algorithms and to do it well. Now that its new owner has publicly committed to doing so, the only thing left is action. That is, if Elon does not fire this team like how he fired the CEO, CFO, and General Counsel.

Trust Over Convenience

In 2013, Ev Williams, one of the founders and ex-CEO’s of Twitter, shared a profound insight that the Internet was just about enabling convenience. Nothing more. That insight fueled the rise of his first startup, Blogger, then Twitter. That insight of maximizing convenience for people to do what they’ve always done for millennia – express themselves, find information, create memes, seek entertainment, troll other people – is at the heart of every social media platform.

While the need for convenience has not changed, what has changed is a stronger demand for trust and transparency. People want to be on a platform to easily socialize and freely express themselves, as long as they feel there is no secret human or machine behind the scenes, manipulating and favoring one person’s expression over another without due process. This concern is at the core of every social media platform. This concern exists because algorithms are secretive and opaque. To fix this concern, the obvious step (without hurting the business, which we’ve shown it won’t) is algorithmic transparency.

When Elon first uttered his desire to buy Twitter and open source its algorithm, a new repository was actually created by someone at Twitter on the company’s GitHub page called “the-algorithm”. That url got a lot of people's hopes up, but it soon disappeared when whether Elon will (or want to) buy Twitter was in doubt.

Now that Elon does own Twitter, he has an opportunity to deliver on his commitment by reviving that url, open source Twitter’s algorithm, push other social media platforms to do the same, still run a great business, and ultimately (no hyperbole here) alter the course of humanity. Whether he keeps this commitment or not ought to be the focus of our attention, not writing poems.

The hope for algorithmic transparency is alive. Let’s hope it is not misplaced on the wrong person.