https://jonathanwarden.com/understanding-bridge-based-ranking/ Jonathan Warden's Blog 1. Home 2. Search 3. About 4. Social Protocols 5. Philosophy 6. Theory 7. Travel 8. Programming Language Design 9. Dark Mode Table of contents 1. Introduction 1. Extracting Information 2. Projection in Opinion Space 3. Why it Works 4. Common Ground 2. Understanding the Algorithm 1. The Intercept is Common Ground 2. The Intercept is not the Average 3. Matrix Factorization 3. Conclusion Featured image of post Understanding Bridge-Based Ranking Understanding Bridge-Based Ranking Jan 01, 2024 11 minute read Introduction Bridge-Based Ranking is an alternative way to score and rank content by adjusting for user polarization. The most successful implementation of Bridge-Based Ranking, X's Community Notes, explains that the algorithm favors notes that are rated highly by users across a "diversity of perspectives". But as I show in this article, ratings from users with diverse perspectives are not necessary for a note to rank highly. It is somewhat more accurate to say that the note must be highly rated regardless of diversity of perspective. The algorithm works by attempting to model why a post receives the votes it does: how many votes are due to users left-wing or right-wing biases, and how many are due to other factors. If a post is only appealing to right-wing voters, and an online forum is dominated by right-wing voters, then that right-wing bias probably explains why it gets so many upvotes. So the algorithm tries to correct for this bias, and estimate how many upvotes a post would receive if there was an equal balance of left-wing and right-wing voters. Extracting Information Now why would we want to do this? Why for example would a predominantly left-wing community want to artificially give right-wing opinions more weight, especially if they think their own side is better-informed? For a fact-checking product like Community Notes, plausible political neutrality may be necessary for public acceptance. But bridge-based ranking has advantages beyond political neutrality: it actually allows us to extract more information from users. In Community Notes, users rate notes as "Helpful" or "Not Helpful". If Community Notes was dominated by leftists, what would we learn by the fact that a note received a lot of "Helpful" votes? That it is helpful? Or that it supports a left-wing worldview? Or both? It's hard to say: perhaps a marginally helpful note that supports a left-wing world view gets more votes than a truly helpful note that supports a right-wing world-view. We can't tell just from the raw vote counts how "helpful" the note is. Bridge-based ranking on the other hands let's us break down the vote counts, attributing some to whatever users think "helpfulness" means and others to polarity. So it is not about giving "both sides" equal weight; by cancelling out the effect of political bias, we can actually extract more interesting information from the users' votes. Projection in Opinion Space The chart below illustrates how this works. This charts shows a subset of notes from the Community Notes public data set, run through my own implementation of the algorithm. The horizontal axis shows the note's "polarity" - e.g. +1 for right-wing and -1 for left wing - and the vertical axis shows its "helpfulness". The note's final score is its vertical component, or its projection on the "helpfulness" axis. The colors of the dots indicate their actual status in Community Notes. Community Notes Polarity Plot (Notes) Notice how there is a large spread along not just the the vertical axis, but also the horizontal axis. If want we want to know how helpful a note is, the horizontal axis is just noise. But there is a lot of information along the vertical axis. Separating the polarity factor from the helpfulness factor by ignoring the horizontal component lets us extract extract this information. But what is this information? It is a measure of some aspect of a post which increases upvotes on that post independently of users political biases. What exactly this feature is is impossible to say, but presumably it reveals how users interpret the idea of "helpfulness". Why it Works People are politically biased, but they are also in a sense biased towards helpfulness. That is, they will mostly upvote notes that support their political perspective but they will especially upvote notes that support their perspective and are actually relevant and factually accurate. And they will tend to downvote notes that support opposing perspectives, but will downvote even more zealously when those notes use false or misleading information. When bridge-based ranking algorithm dissects users voting behavior and factors out the polarity component, it finds that most users are at least somehat biased towards helpfulness! You can see this in the plot of a sample of Community Notes users below. Community Notes Polarity Plot (Users) There is clump of users in the upper-right quadrant because community notes users are overall right-leaning. But notice also that the helpfulness factor for these users is mostly above zero. They are also mostly biased towards helpfulness. These users are more likely to upvote posts that support a right-wing worldview, and also more likely to upvote posts that are helpful. Common Ground This vertical component in these plots represents common ground. It is something users tend to agree on independently of their politics. In the case of Community Notes, this is presumably some common idea of what constitutes "helpfulness". But in general what exactly the common ground is depends on the community. Suppose for example there is a forum for Harry Potter fan fiction that unfortunately in recent years it has been overwhelmed by debates about whether J.K. Rowling is transphobic. There is still a lot of good fan-fiction being posted, but the home page is dominated by posts about the controversy. In this case, the horizontal axis would likely represent the pro- and anti- J.K. Rowling factions, and the vertical axis would represent the common ground of the community: quality Harry Potter fan fiction. Using bridge-based ranking we can in a sense de-polarize the forum, factoring out the effect of polarization and getting back to community's original essence. Politics is not the only factor that can divide a forum. Suppose there is a popular forum for posting ridiculously cute pet pics. Sadly, in recent years, two factions have formed: the cat faction and the dog faction. The more extreme cat people mercilessly downvote pictures of dogs (regardless of how cut they are), and the dog people vice versa. Recently, the dog faction has gained the upper hand, and a cat-picture has little chance of making the front page, no matter how frigging adorably it is. Again, by separating the dog-cat factor from the common ground factor, we can re-focus the community on it's original purpose: raw frigging cuteness. Understanding the Algorithm But how does the algorithm actually work? How does it determine the polarization factor and common ground factor for each user and post? It actually works using a fairly simple algorithm called Matrix Factorization. Below I will explain how the Matrix Factorization algorithm works, starting with the version implemented by Community Notes and described in the Birdwatch Paper. There is also a good writeup by Vitalik Buterin. In my next post describe my variation of the algorithm that uses 2D matrix factorization. A good way of understanding Matrix Factorization is that it is like running a bunch of linear regressions: one for each user and each item. For example, suppose we have already discovered the polarity factor for each user, and we want to find the polarity factor for each post. A linear regression predicts users' votes as a function of their polarity factors. For a highly polarizing right-wing post, the regression line might have a positive slope: Highly Polarizing Right-Wing Post Vote +1 | / | / -1 ______|/______ +1 User's Polarity Factor /| / | / | -1 In this chart upvotes have a value of +1 and downvotes have a value of -1. All the right-wing users upvoted and all the left-wing users downvoted (as shown by the little s). So the best fit is a line with a slope of approximately +1: the more right-wing the user, the higher the probability of an upvote, and the closer the predicted value is to 1. The more left-wing, the higher the probability of a downvote, and the closer the predicted value is to 0. Note that there are more right-wing users than left wing users, but it doesn't make a difference. Even if there were 100 right-wing users and 2 left-wing users, the slope of the best fit would be approximately the same. This is why bridge-based ranking does not favor the majority. A very polarizing lift-wing post might have a negative slope: A Highly Polarizing Left-Wing Post Vote +1 \ | \ | -1 _____\|________ +1 User's Polarity Factor |\ | \ | \ For a completely non-polarazing post, on the other hand, the slope would be zero: A Non Polarizing, "Good" Post Vote +1 - - -|- - - | -1 ______|________ +1 User's Polarity Factor | | | This is a good post. Not just because the upvote probability is independent of the user's politics, but because this post receives mostly upvotes - the intercept is above zero. This post has some quality that users of this forum are looking for. Now, suppose there is a post that looks like this: A "Good" but Polarizing Post Vote +1 | / |/ /| -1 __ /__|________ +1 User's Polarity Factor / | / | This post has a positive slope, so it is clearly very polarizing. But the positive intercept means that voting behavior for this post cannot be explained entirely by politics. There is also a component that makes users more likely to upvote it independently of politics. The Intercept is Common Ground So the intercept represent "common ground". It represents something about a post that causes users to upvote independently of politics that cannot be explained entirely by users' polarity factors. The Intercept is not the Average We might suppose that the last post above will receive more upvotes than downvotes because it has a positive intercept. But this is not necessarily the case. It depends on how many left-wing and right-wing users there are. The intercept is not the average: a post can have a positive intercept even though it receives more downvotes than upvotes, or it can have a negative intercept even though it receives more upvotes than downvotes. What a positive intercept does tell us is that this post would receive more upvotes than downvotes if there was an equal balance of left and right-wing users. It also tells us how users would hypothetically vote if they were all totally a-political. In such a hypothetical world, the only thing influencing users votes is some common-ground factor that aligns with the intent of this particular community, attracting upvotes independently of politics. Matrix Factorization Okay, so we have used regression analysis to find the polarity factor for each post (the slope) of the regression line. But in order to do these regressions, we first need to know the polarity factors for the users. But how do we find these? Well, if we knew all the posts' polarity factors, we could use regression analysis to estimate the probability that a user upvotes a post as a function of the polarity factors of the posts. The slope of the regression line would then be the user's polarity factor. The regression line for a very right-wing user, for example, might look similar to that for a very right-wing post. A Right-Wing User Vote +1 | / | / -1 ______|/______ +1 Post's Polarity Factor /| / | / | -1 But we seem to have a chicken-and-egg problem, where we can't find the polarity factors of users unless we know the polarity factors for posts, and vice versa. However, the Matrix Factorization algorithm solves this by discovering the polarity factors (and intercepts) for every user and every post all in one go. It does this by using a single equation to estimate the probability that user $i$ upvotes post $j$: $$ y_{ij} = w_ixx_j + b_i + c_j $$ Here $w_i$ is the user's polarity factor, $x_i$ is the post's polarity factor, $b_i$ is the user's intercept, and $c_j$ is the post's intercept. It then simply finds a combination of values for every $w_i$, $x_j$, $b_i$, and $c_j$ that best fits the data - that produce values for $y_{ij}$ that are closet to the actual values of users vote ($y_{ij} $). This is usually done using a variant of the standard gradient descent algorithm. The polarity factor the algorithm discovers doesn't necessarily correspond exactly to politics, or cat-dog preferences, or any measurable quantity. It may be a linear combination of factors. But whatever it is, it represents some latent factor of users and posts that does a good job predicting their votes. Conclusion One of the reasons for my interest in bridge-based ranking is that I think it may be a critical part of a social protocol for a self-moderating community. Without it, user polarization will tend to lead to either suffocating uniformity or least-common-denominator mediocrity. Bridge-based ranking can be used in any forum with high entropy (lots of downvotes) as a way to identify posts with posts with high Information Value based on the common-ground factor. In my next article, I discuss ways that this algorithm can fail, and introduce an improved implementation of the algorithm that users 2-dimensional matrix factorization. (c) 2016 - 2024 Jonathan Warden's BlogBuilt with Hugo Theme Stack designed by Jimmy