In 2016, I headed a compact team at Instagram that conceived and launched one of the largest content-distribution experiments ever run: rolling out a personalized ranking algorithm to the platform’s (then) 500 million users. Expecting backlash, we spent the following years rigorously measuring the differences between people shown this changing “recommendation algorithm” (as it’s sometimes called) and a small randomly selected group still seeing the reverse-chronological feed used since Instagram began.
Those differences pointed to a dramatically better experience with the new algorithm across every part of the app.
Although I still believe algorithmic ranking is the right choice for social platforms, it comes with drawbacks. A few are worth naming: more platform influence over content distribution, unclear decision rules, the danger of surfacing harmful material, and ordinary user annoyance. Those drawbacks recently prompted Twitter’s potential future owner, Elon Musk, to urge “open sourcing the algorithm.”
From an engineering standpoint, that notion seems far too simple, because open sourcing a machine-learning model tells us very little about its consequences. Yet the demand for transparency is legitimate, and it can start with revealing experiments like the one I ran at Instagram. I would argue that meaningful transparency lives in open-source experimentation, not algorithms.
I am not suggesting what ought to be done with the results of open-source experimentation; instead, this piece is a beginning point for thinking about transparency in the setting of modern ranking systems. Here, I explain why experimentation is both necessary for algorithmic ranking and a better target for future attempts to make social-media content distribution less mysterious.
Modern algorithms favor the “most interesting” content
Most social platforms have far more content than any person could sensibly take in.
Instagram started in 2010 with a reverse-chronological feed, which puts the newest “connected” content (meaning content from people you decide to follow) at the top of a user’s feed. Six years later, the average user was only seeing 30% of their connected content. Since attention spans are limited, we concluded that this amount reflected the natural ceiling of what an average person wanted to consume. The point of adding algorithmic ranking was to turn that 30% into the most interesting content rather than the newest. Other platforms like TikTok, YouTube, and Twitter have their own ratios (i.e., they make different amounts of content available), but the underlying method is the same: choose the most interesting content within a fixed attention span.
Exactly how a ranking algorithm allocates content defines what “most interesting” means. One approach is to keep it unpersonalized — everyone eligible to view the same content sees it in the same sequence. Algorithms designed to put the most-liked content first, or surface the prettiest photos, or even showcase “editor’s picks” all belong in that group. But taste is deeply personal; two users following the same people will still want different things. Unpersonalized ranking cannot capture “most interesting” at the scale of billions.
Modern ranking algorithms, by contrast, are personalized: the algorithm chooses different content based on who is browsing. We cannot read a user’s mind or know their exact preferences, but a machine-learning model can use prior behavior to estimate answers to questions such as, “If you saw this content, what is the likelihood you would like it, comment on it, share it, watch it, skip it, or report it?”
Algorithmic ranking blends these predictions with substantial business logic (e.g., broadening content, weighting against hateful content, elevating content from lesser known accounts) to create the foundation for deciding the most interesting content for a specific user.
Why “open sourcing” the algorithm doesn’t work
My understanding of what supporters of open-source algorithms imagine is this: If we release the internal source code and weights of the machine-learning models used in ranking, then engineers, analysts, and others will be able to see why certain content is promoted or pushed down. The reality is that even total visibility into models still reveals very little about their effects.
Machine-learning model predictions change depending on the user, the content, and the situation. Those differences are organized into “features” that a machine-learning model can use to make a prediction. Examples of features include: recent content a user’s consumed, how many of a user’s friends liked something, how often a user engaged with a certain person in the past, and the engagement per view of people in a user’s city.
The calculation behind “net good” — not the micro-details of a particular ranking algorithm — determines if an experiment is successful.
Modern algorithmic ranking models consider millions of these features to generate each prediction. Some models rely on many sub-models to assist them; some are retrained in real time to keep up with changing behavior. These algorithms are complicated to interpret, even for the engineers building them.
The scale and sophistication of these models make it impossible to fully grasp how they arrive at predictions. They contain billions of weights interacting in subtle ways to produce a final prediction; studying them is like expecting to understand psychology by looking at individual brain cells. Even in academic environments with long-established models, the science of interpretable models is still in its early stages. The few current methods for understanding them depend on the privacy-sensitive datasets used during training. Open sourcing algorithmic ranking models would not alter that.
When does an experiment produce a “net-good” change?
Engineers like me measure predictive performance. Rather than trying to understand the internal mechanics of algorithms, we run experiments and watch the outcomes. Ranking teams (usually a combination of data scientists, engineers, product managers, and researchers) may have thousands of simultaneous experiments (A/B tests) that each expose groups of people to versions of ranking algorithms and machine-learning models.
The central question behind an experiment is whether a change is — to use a term I coined — “net good” for the ecosystem. During the rollout of algorithmic ranking to Instagram users, we saw major gains in product engagement and no meaningful changes in reported quality of experience. Once a team decides an experiment produces a net-good change, as we did, it becomes the platform’s default user experience and quietly affects the content that hundreds of millions of people see each day.
Determining net good requires examining the effects of experiments through summary statistics about changing user behavior and content distribution (i.e., which kinds of content are promoted and demoted). For instance, a team can examine how often users open an app or “like” content, how much time they spend on the app per day or per session, how often someone says they are having a “5 out of 5” experience, whether “small” creators are favored over “large” ones, the prevalence of “political” content, and so on. Summary statistics come from processing huge volumes of individual user actions — you are in the test group, you logged on at 3 p.m., you saw your best friend’s video and then liked it, you missed another post by a celebrity, etc. and easily number in the thousands. Teams look for statistically significant differences in those statistics between test and control groups.
It is not enough to say “open-source all the data” — that’s an innovation and privacy nightmare. But it is possible to safely disclose more than companies do today.
Any well-run algorithmic ranking team has a method for deciding whether a change is net good relative to an established baseline. That method may be written down: Anything that increases the number of active users is net good. Or it may rest on judgment: If person X approves after reviewing summary statistics, it’s net good. Or it may be adversarial: If no team can identify a problem, it’s net good. In practice, it may be a blend of all of these.
The calculation behind net good — not the micro-details of a particular ranking algorithm — determines if an experiment is successful. Experiments direct the success of ranking teams inside a company. And the success of ranking teams directs how content is distributed for all platform users.
Because net good is such a consequential label, it makes sense to ask for open sourcing in experiments.
What open source means for experiments
The issue with our current setup is that only the people conducting experiments can examine them. Although there are valid reasons for that, the people making ranking changes are not necessarily motivated to look for certain problems in the way the wider community might be. (Indeed, this is something the open-source movement in software has historically done well — i.e., drawing on a community of engineers to identify problems and add improvements, alongside the core developers working on the project.) By giving the community greater visibility into experiments, the teams running them can set best practices for decision-making and surface effects from experiments that go beyond what the team is studying.
In opening sourcing experiments we need to weigh two competing interests: retaining enough proprietary information for companies to keep innovating while revealing enough to support outside understanding. It is not enough to say “open source all the data” — that’s an innovation and privacy nightmare. But it is possible to safely reveal more than companies do today. Disclosures could happen in two ways:
- Open-source methodology: What is the purpose of ranking changes? What team goals and decision-making can safely be disclosed without hurting company innovation? Open-source experimentation: What are the effects of ranking changes? What information can be shared so that third parties such as auditing agencies can inspect the effects of ranking experiments without giving up user privacy?
Disclosure by itself does not resolve the broader incentive problems in algorithmic ranking. But it gives the wider community a well-informed foundation for thinking about them, and it directs research and attention to where it can do the most good.
Open-source methodology
It is important to keep in mind that the major decision in algorithmic ranking is what counts as a net-good change. Promoting open-source methodology creates more visibility into how those choices are made and how platforms assess their content ecosystem. The data involved would already be summarized, which removes concerns about infringing individual privacy. The risks of disclosure, then, are mostly about competitive advantage and bad actors such as spam farms and coordinated attackers. To begin, here are three kinds of info that would not be risky for a platform to share:
- The broad process for deciding whether a new ranking variant is a net-good change. Who, if anyone, holds decision-making power over broader algorithm changes. An explanation of summary statistics available in decision-making and evaluated in experiments.
A hypothetical disclosure using that information might look like this: Each year, a platform’s executive team sets targets for engagement measures, plus secondary targets tied to content quality. The ranking teams responsible for meeting the targets are allowed to run up to 1,000 experiments a year, each involving millions of users. A product manager must review the experiments before they start, and meets once a week with the relevant ranking teams to review the ongoing impact on the primary and secondary targets, along with any other effects that emerge as statistically significant, such as content shifts to larger accounts or the prevalence of politically tagged content. Then, the final decision on whether or not to ship an experiment rests with the executive team. The ranking teams assess the overall contribution of algorithm updates by running one experiment that “holds back” all changes over the year.
The key question in experimentation transparency is: How can we share experiment data more broadly without giving up privacy?
That kind of disclosure helps us see how decisions are made inside a company and could be recorded in platform transparency centers and annual reports. More detailed disclosures, which provide more valuable insight into decision-making, are also more likely to risk revealing company secrets. These disclosures would include more about the purpose of summary statistics, such as:
- Which summary statistics are desirable, which are undesirable, and which serve as guard-rails (and should not change). Exact formulas used to judge whether a decision is net good. Lists of all experiments with hypotheses, dates, and decisions made.
Whether this is too detailed for a disclosure is open to debate and depends on the specific circumstances and goals for each product. But returning to the Twitter example and the much-discussed “spam” problem, here is a hypothetical scenario showing a useful disclosure: Suppose Twitter ran 10 experiments aimed at reducing spam prevalence. Each experiment was meant to test whether changing the predictor of “clicking into a tweet” would cut the number of users seeing spam. In those experiments, lower spam reports were treated as a desirable outcome, lower replies were undesirable, and the number of retweets were used a guard-rail and expected to stay stable. Experiments one to five used larger, re-trained models predicting if a user would “click into a tweet.” Experiments six through 10 left the model unchanged but reduced the weight of click predictions in final ranking. The current production ranking model was used as a control group. All experiment variants started on May 20, involved experiment groups with 5 million users each, and ran for two weeks. Experiment seven, with a moderate decrease in weight, was approved by the product manager on June 10 and became the baseline experience.
A disclosure like that would help outsiders judge whether Twitter is both actively trying to solve the spam problem and doing so with a sound approach. Transparency creates a risk of bad actors using information to adjust tactics, but it also makes ranking teams more accountable to their users and builds more trust in how the user experience unfolds.
Open-source experimentation
While open-source methodology gives insight into a ranking team’s intent, it does not let outside parties understand the unintended consequences of ranking decisions. For that, we should look at open sourcing the experiment data itself.
Studying experiments requires access to confidential information that is only available to employees, such as individual user actions, e.g., “User A saw this video, watched it for 10 seconds and then liked it.” Comparing summary statistics for this information between test and control groups lets the company understand the algorithmic changes it makes. The key question in experimentation transparency is: How can we share experiment data more broadly without giving up privacy?
The most transparent form of open-source experimentation would mean releasing the raw information — every single person’s action in every experiment ever run. With that, outside parties could reach proper, scientific conclusions about user behavior and content shifts in social media. But this is a naive goal. Individual user actions are sensitive and personally revealing, and in some contexts they even put lives at risk. We should instead aim for a level of transparency that does not expose sensitive information or violate consent, yet still allows other parties to study the results of experiments scientifically.
- Limit the audience: Share raw experiment data to a smaller trusted group outside the company, such as a set of third-party algorithmic auditors that could be bound by professional regulations. Individual disclosure: Allow users to see every experiment they have been exposed to. Individual opt-in: Reduce some privacy concerns by letting individuals choose to disclose their actions to specific groups, such as by allowing opt-in into monitored academic studies through in-app mechanisms. Summarization: Publish less sensitive information by bucketing experiment data into cohorts (e.g., disclose shifts in content distribution toward larger accounts, videos, specific countries, etc.).
These approaches all hand the tools of analysis to people who do not work at social platforms and therefore are not shaped by company incentives. If we look back at the multi-year experiment I led on introducing Instagram’s ranking algorithm, having fresh eyes on the experiment group could have raised new perspectives on concerns such as whether ranking causes a filter bubble, whether introducing ranking causes a shift toward more political accounts, and whether people post more harmful content as a result of ranking. Without access to data, we are all left with faulty reasoning based on headlines and anecdotes.
***
Despite the spread of algorithmic ranking models, their inner workings are not well understood — nor is that the point. Companies study the effects of algorithms by running experiments to decide whether the changes they produce are net good for their content ecosystems.
Today, outside parties, including the users who interact with these products every day, have no way to reach conclusions about what is net good because experiment data is private and the decision-making methodology is not disclosed. That does not have to be the case: It is possible to open up more of the decision-making methodology while still preserving companies’ ability to compete. Information about experiments can be disclosed in a way that lets outside parties draw conclusions without sacrificing privacy.
Transparency is itself a quality, but real transparency is the stronger aim. Going forward, let’s concentrate more openly on revealing experiments, not algorithms.