Observation: Julia Angwin and Jeff Larson left their jobs at ProPublica to “start a crazy adventure.”
Hypothesis: Their recently unveiled organization, The Markup, is setting up a new model for newsrooms to report on the societal effects of technology, using the scientific method (as seen, well, here in this lede).
Data/evaluation/findings: TK.
Angwin and Larsen, a journalist-programmer team at ProPublica, had uncovered how Facebook let users target ads at “Jew haters” and enabled advertisers to exclude certain races and ages from housing and job ads, among other investigations into how algorithms are biased. The work even earned Angwin a public shoutout from Facebook:
They split from ProPublica in April — bringing a couple staffers with them on their way out, and roping in cofounder and executive director Sue Gardner, formerly of the Wikimedia Foundation and the Canadian Broadcasting Corporation — and now are ramping up said crazy adventure:Thanks @JuliaAngwin. You've done a lot to uncover issues in our ads systems, which we've worked hard to fix. This new API is an important step towards greater transparency + other changes we're making like view ads and the archive going back 7 years for all political & issue ads. https://t.co/z7wagLZRyV
— Facebook (@facebook) May 11, 2018
The Markup is a nonpartisan, nonprofit newsroom. We produce meaningful data-centered journalism that reveals the societal harms of technology. We hold the powerful to account, raise the cost of bad behavior and spur reforms. Our journalism is guided by the scientific method; we develop hypotheses and collect data to test those hypotheses.
Funded by $20 million from Craig Newmark Philanthropies, $2 million from the Knight Foundation (which, disclosure, has supported Nieman Lab in the past), and $1 million from a collection of other journalism philanthropy organizations, The Markup kicks off in early 2019. I spoke with Angwin (with a short interlude on the financial details from Gardner) about the organization’s plans, its distinction from ProPublica, and how others can get involved going forward. Our chat has been edited and condensed for clarity.
ProPublica is literally the best job in journalism. It was and remains the best job in journalism. We took a crazy leap. It was crazy to walk away from those jobs, and I’m so happy it turned out to be a good thing for us. But it was a huge, huge risk. Nothing about what we did reflects poorly on them. It’s more about: We have an idea about how journalism should be. It’s much more tech-focused than any newsroom, even though ProPublica is the most tech-infused newsroom out there. We want to take it to another level.
It doesn’t have to be ‘he said, she said.’ It’s more about: this is the amount of evidence to support this hypothesis, and then here are the limitations of this. There are always limitations to our findings. Even though climate change is well accepted scientifically, there are limitations for those findings as well. That’s our goal, to try to frame our journalism around that.
What that means in practice is having people with technical and statistical skills involved in an investigation from the outset. So much of what happens in traditional newsrooms, in every newsroom I’ve ever worked in, is that there’s a data desk. A reporter goes over to the desk and basically orders data like it’s a hamburger. Usually by then, the reporter has already done the reporting and has a hypothesis based on the anecdotes. Then, if the data doesn’t support it, there’s a fight between them and the data desk. Or, more often, there’s not even data available.
There isn’t data about most of the important questions we need answered as a society. The reason there’s no data about them is that there’s no political will for it. The reason we don’t know what happened on Facebook during the elections, for instance, is because Facebook would have to tell us — and why would they want to? It’s important to start the investigation earlier, with ‘What is the question we want to ask?’ and ‘What is the best way to get that data?’
Of course, traditional reporting is one of those ways. It’s not a good idea to just wade into a topic you don’t know anything about. You have to talk to people and understand what you’re talking about. But at the same time, I think it’s really important for journalists to provide data because data is how we as a society make decisions. We have chosen to be, pretty much, a scientifically driven society, and we appreciate data. Mostly, we still agree that facts matter. For journalists, the more data we can bring to the table, the more we can say ‘Hey, this is what we have found.’ It’s not just three anecdotes — it could be 10 or 10,000. The fact is, we need to bring a bigger sample size to the table.
The thing that we’re looking for is not necessarily heavy programming skills. We will need some of that, but there’s a really interesting dynamic I’ve noticed doing this type of work. It’s more about being open to the scientific method — being open to the idea that we’re going to let the data guide us and we’re going to go find the data for important questions as a way of doing journalism. It takes a certain kind of mindset to be open to that. There are lots of other ways to do journalism; I’m not saying this is the only way to do it. This is one way that I want to do it. I have noticed that people who have nontraditional backgrounds can be really good at this kind of thing. I have a feeling that we will have a wider variety of people with maybe not as often a traditional journalism background.
Two of the people I took with me from ProPublica, Surya Mattu and Madeleine Varner, are both programmers, but they’re self-taught and they both studied art. They’re both artists primarily, but they have that investigative mindset and curiosity. It’s a pretty nonraditional hire but those are the kinds of people who have worked out really great.
In general, in our investigations, we often reach out to academic and subject matter experts for advice. We’re not actually statisticians. We know we are just amateurs. We always reach out to experts on a case by case basis for advice on investigations. We may formulate that into an advisory group, but we haven’t decided yet. Of course people can donate! I know we did receive an enormous gift and we are so grateful, but of course if we want to run this — that’s about four years of funding, so they can contribute to the fifth year!
[Here’s that interlude with Sue Gardner, The Markup’s cofounder and executive director, with more on the finances.]
The other thing we’re going to be working on is trying to find a sustainable model for that kind of journalism. Investigative journalism is the most expensive kind of journalism. Data-centered investigative journalism is even more expensive. It’s a niche product. It’s not broadly appealing to large numbers of people. Journalism is best when it’s paid for by the users. Then all the incentives are virtuously aligned.
What we plan to do is exactly what I did at the Wikimedia Foundation. When I went there, we did not have a business model. We were a nonprofit, but we weren’t bringing in a lot of money. My first job was to figure out sustainability for Wikipedia. When you look at it now, it seems really obvious for what the model for Wikipedia should be, but it was not obvious in 2007. We deliberately set out to spend two years experimenting with different revenue model. We solicited major gifts, we spoke with foundations and got grants for the organization, we did what we called the ‘many small donors’ model, and we experimented with various kinds of earned income. I had always hoped when we started that the ‘many small donors’ model would be successful, and it did turn out to be successful.
I want to do the same thing with The Markup. We’re going to experiment for probably around two years and we’re going to double down on what seems to be working. That is what Craig Newmark’s money and the major grantmakers’ money has bought us — that runway, so we have a couple years to experiment and we have some time to figure out what will work in the longterm.
[Okay, back to Julia.]
I really, really am excited to try to build a model for a new way for doing tech-driven journalism. There was a time when journalists really didn’t know anything about business, and then there was the effort to educate journalists about finance and I was part of that. I got the Knight-Bagehot fellowship at Columbia and I ended up getting my MBA because I was a business journalist and I wanted to have that expertise in the area I covered. I feel like we need that in technology.
Technology is invading every part of our lives, and it is used as a cover for political decisions. Journalists in every field need to have more skills to investigate those types of decision-making that is embedded in technology. I’m hoping we build a model that is replicated.
I see us as a FiveThirtyEight — when FiveThirtyEight started, they were the first ones doing major meta statistical analysis of polls and using that to inform political coverage. Then everybody copied it, like with The Upshot. Part of their success was extended through invitation. I hope that happens to us. I still want us to exist, I don’t want to be copied out of existence, but if we can build a model about how you can do this kind of work that’s journalists paired with technologists and expertise — I would be thrilled if that spread to other newsrooms and became a practice and a field.