We're like, 'Well, what do we do? We're going to put this out there. Everyone's going to have all these questions. They're probably going to be really skeptical, and we know it might be a total dumpster fire.'
Keith Coleman & Jay Baxter
Keith Coleman (VP of Product, Community Notes) & Jay Baxter (Founding ML Engineer, Community Notes)
28 quotes across 1 episode
An inside look at X's Community Notes
If I were to start a company in that company, it would be even leaner than I would've made it before. I've been amazed with just how much the team is able to accomplish with a small group and I think because of a small group-
One philosophical thing that's important is that we want all of humanity to participate and sometimes people are surprised by that. We have all of humanity. We then have the data to understand what notes will be helpful to actual humanity.
Community Notes adds additional context. It's not fact-checking necessarily, right? So there are cases where the post could be true. But maybe, it's just misleading because there there's no context or there's missing context.
We have the philosophy that users should be able to make up their own minds, right? Like, 'Here's extra context, take it or leave it,' right?
I think one reason perhaps the product has done so well and survived is the nature of the product itself. It is designed to produce information that is found helpful by people who normally disagree. And so even if you have CEOs or leaders who might disagree, there's a good chance actually they'll find it helpful.
We actually look for agreement from people who have disagreed in the past. And what we see is when people actually have that sort of surprising agreement, that's what makes the notes so neutral and accurate and well-written, really, overall.
We found a few things. One, people were hesitant to write a note on a controversial topic because they didn't want to get attacked or harassed online... Two, and this is super interesting, people are actually more willing to cross partisan boundaries when they are anonymous or pseudonymous than when they are under their real name.
Society often feels really polarized, you hear people talk about it all the time, no one can ever agree on anything, but actually Community Note shows you people really can agree on quite a lot. Even on super controversial topics related to politics and everything, there's a lot of agreement, that's why notes work.
I kept coming back to this problem. I'm like, 'Man, how is the world going to deal with this information quality issue of what we get on social media?' Wherever get it. I'm at this company where you can make a difference on this problem, why not go and try some crazy ideas and see if one of them might work?
For me, this project, I feel like I get to do community service with this project. I see my work as in service of the people and the community, and that's what motivates me. The only thing that I care about is delivering the outcome that the world finds helpful.
If I were to start a company and had to change the way I think about starting that company, I would be even leaner than I would've made it before. I've been amazed with just how much the team is able to accomplish with a small group.
The team did not get distracted by much all through the period during which the acquisition happened. There was a lot of opportunity for distraction. This team was shipping every week, we were super focused on the goal, let's make this thing work, let's get these notes out there.
The code that decides what notes share has to be out in the open. All of the data and ratings that make it happen have to be out in the open. People should be able to take the code and data and replicate the whole service.
We don't have a button that will change the status of a note. So if a note is showing because the people have rated it and found it helpful, it is going to show. We can't change that.
Every post is eligible for notes. We shouldn't exempt Elon. We shouldn't exempt government figures. We should be like everyone... Even advertisers can get notes.
Deleting code is more important than writing it a lot of the time. So I think so often maybe due to promotion incentives or just regular human tendency, engineers have a tendency to add these little incremental wins that actually add more of a long-term maintenance cost than is clear.
I kept coming back to this problem. I'm like, 'Man, how is the world going to deal with this information quality issue of what we get on social media?' Wherever get it. I'm at this company where you can make a difference on this problem, why not go and try some crazy ideas and see if one of them might work?
Look at Community Notes for example. If I had stayed running a large consumer PM team, what would I have produced? 16 more pages of OKRs? I don't know, a bunch of documents? I think building Community Notes has had way bigger impact on the world.
I think there's one myth that can get in people's ways. The idea that the more people you manage or the larger your scope is, the more impact you have. I definitely do not think that is true.
There are a number of principles that I think when we first shared them with people at the company seemed maybe a little bit crazy. But I think they are the reason the product works, and I think they've been very important, and we do. We come back to them regularly, today, all the time.
This thing is going to be the voice of the people. It's going to represent the voice of people. It's not going to represent the company's voice. So it is not a tech company deciding what shows. It is the people deciding what shows.
If there's a problem with a note that's so bad, you want to do something about it's a problem with the system. We need to redesign the system to be showing good notes.
We were very disciplined, I guess you could say, about having the product prove itself at every given point.
Anytime that we were proposing doing something with the product, like running some research test, or running the pilot, or expanding the pilot, we always had the data that had convinced us that that was a good decision.
One key attribute is there's one clear driver of the project, who's effectively a founder. I guess maybe you could have two or something, but really clear, there's driver of the project and also there's one clear decision-maker that they go to.
It's actually really hard from an end perspective to actually open source the actual algorithm that's running on the actual data. Because the way large-scale services like this are usually architected does not naturally lend itself to being run as a script by someone who's downloaded a TSV.
We actually look for agreement from people who have disagreed in the past. And what we see is when people actually have that sort of surprising agreement, that's what makes the notes so neutral and accurate and well-written, really, overall.