Nieman Foundation at Harvard
Are you willing to pay for Prepare to be asked before year’s end
ABOUT                    SUBSCRIBE
Sept. 24, 2018, 2:55 p.m.
Reporting & Production

Watch out, algorithms: Julia Angwin and Jeff Larson unveil The Markup, their plan for investigating tech’s societal impacts

“Journalists in every field need to have more skills to investigate those types of decision-making that are embedded in technology.”

Observation: Julia Angwin and Jeff Larson left their jobs at ProPublica to “start a crazy adventure.”

Hypothesis: Their recently unveiled organization, The Markup, is setting up a new model for newsrooms to report on the societal effects of technology, using the scientific method (as seen, well, here in this lede).

Data/evaluation/findings: TK.

Angwin and Larsen, a journalist-programmer team at ProPublica, had uncovered how Facebook let users target ads at “Jew haters” and enabled advertisers to exclude certain races and ages from housing and job ads, among other investigations into how algorithms are biased. The work even earned Angwin a public shoutout from Facebook:

They split from ProPublica in April — bringing a couple staffers with them on their way out, and roping in cofounder and executive director Sue Gardner, formerly of the Wikimedia Foundation and the Canadian Broadcasting Corporation — and now are ramping up said crazy adventure:

The Markup is a nonpartisan, nonprofit newsroom. We produce meaningful data-centered journalism that reveals the societal harms of technology. We hold the powerful to account, raise the cost of bad behavior and spur reforms. Our journalism is guided by the scientific method; we develop hypotheses and collect data to test those hypotheses.

Funded by $20 million from Craig Newmark Philanthropies, $2 million from the Knight Foundation (which, disclosure, has supported Nieman Lab in the past), and $1 million from a collection of other journalism philanthropy organizations, The Markup kicks off in early 2019. I spoke with Angwin (with a short interlude on the financial details from Gardner) about the organization’s plans, its distinction from ProPublica, and how others can get involved going forward. Our chat has been edited and condensed for clarity.

Christine Schmidt: How far along was The Markup when you left ProPublica?

Julia Angwin: I left ProPublica with the hope and dream of doing something like The Markup. It wasn’t appropriate for me to try to raise money while I was there because they’re also a nonprofit. I had to leave without any money. I didn’t know if I’d be able to raise money. It was terrifying. Jeff and I just figured we’ll try this and hopefully someone will hire us if we fail. We had been talking about it for a while because we wanted to expand our work. We had a team of him, me, a halftime programmer, and a sometimes researcher. We loved our work but we wanted to do more of it with more people. So we decided to jump off the cliff and hope it worked out.

Schmidt: Can you say more about why you decided to build this organization separate from ProPublica and the work you’d already been doing there?

Angwin: ProPublica was great. Jeff and I had a great run there — we had so much fun and were able to establish and pioneer this type of reporting that we did together with a journalist and programmer working together from the beginning. Our dreams, though, were pretty ambitious. We wanted a newsroom, and we will have a newsroom of 20 people. That would be a significant commitment for ProPublica. We discussed with them about doing it internally and we all agreed in the end that it would be better to go off and do it on my own. It was something we were all feeling sad about, but it wasn’t angry.

ProPublica is literally the best job in journalism. It was and remains the best job in journalism. We took a crazy leap. It was crazy to walk away from those jobs, and I’m so happy it turned out to be a good thing for us. But it was a huge, huge risk. Nothing about what we did reflects poorly on them. It’s more about: We have an idea about how journalism should be. It’s much more tech-focused than any newsroom, even though ProPublica is the most tech-infused newsroom out there. We want to take it to another level.

Schmidt: What is that next level? What are the nuts and bolts of how this organization will operate differently?

Angwin: We describe ourselves as doing journalism that is based on the scientific method. The idea is that objectivity has been a false god for journalism for a long time. It started out as a decent idea, but it’s led to a lot of what people call false equivalents. I think journalism needs a new guiding light, a new philosophical approach, and I think that approach should be the scientific method. What that really means is we develop a hypothesis. Maybe the hypothesis is: ‘Brett Kavanaugh. Did he actually harass a woman or not?’ Then you collect evidence — how much evidence is there for and against this. Then you describe the limitations of your evidence: ‘So far the evidence is only one/two people.’

It doesn’t have to be ‘he said, she said.’ It’s more about: this is the amount of evidence to support this hypothesis, and then here are the limitations of this. There are always limitations to our findings. Even though climate change is well accepted scientifically, there are limitations for those findings as well. That’s our goal, to try to frame our journalism around that.

What that means in practice is having people with technical and statistical skills involved in an investigation from the outset. So much of what happens in traditional newsrooms, in every newsroom I’ve ever worked in, is that there’s a data desk. A reporter goes over to the desk and basically orders data like it’s a hamburger. Usually by then, the reporter has already done the reporting and has a hypothesis based on the anecdotes. Then, if the data doesn’t support it, there’s a fight between them and the data desk. Or, more often, there’s not even data available.

There isn’t data about most of the important questions we need answered as a society. The reason there’s no data about them is that there’s no political will for it. The reason we don’t know what happened on Facebook during the elections, for instance, is because Facebook would have to tell us — and why would they want to? It’s important to start the investigation earlier, with ‘What is the question we want to ask?’ and ‘What is the best way to get that data?’

Of course, traditional reporting is one of those ways. It’s not a good idea to just wade into a topic you don’t know anything about. You have to talk to people and understand what you’re talking about. But at the same time, I think it’s really important for journalists to provide data because data is how we as a society make decisions. We have chosen to be, pretty much, a scientifically driven society, and we appreciate data. Mostly, we still agree that facts matter. For journalists, the more data we can bring to the table, the more we can say ‘Hey, this is what we have found.’ It’s not just three anecdotes — it could be 10 or 10,000. The fact is, we need to bring a bigger sample size to the table.

Schmidt: So what are you looking for in different journalists and programmers? Who will make up that team?

Angwin: We’re going to put up really specific job descriptions, but I can talk about it on a high level.

The thing that we’re looking for is not necessarily heavy programming skills. We will need some of that, but there’s a really interesting dynamic I’ve noticed doing this type of work. It’s more about being open to the scientific method — being open to the idea that we’re going to let the data guide us and we’re going to go find the data for important questions as a way of doing journalism. It takes a certain kind of mindset to be open to that. There are lots of other ways to do journalism; I’m not saying this is the only way to do it. This is one way that I want to do it. I have noticed that people who have nontraditional backgrounds can be really good at this kind of thing. I have a feeling that we will have a wider variety of people with maybe not as often a traditional journalism background.

Two of the people I took with me from ProPublica, Surya Mattu and Madeleine Varner, are both programmers, but they’re self-taught and they both studied art. They’re both artists primarily, but they have that investigative mindset and curiosity. It’s a pretty nonraditional hire but those are the kinds of people who have worked out really great.

Schmidt: The reception I’ve seen on Twitter is a lot of excitement and a lot of interest. What are other ways that people could get involved or partner with The Markup?

Angwin: We’re going to have the ProPublica model with Creative Commons licensing of our stories, so they will be widely republishable. Also, just like ProPublica, we’re going to have partnerships with big media outlets for our big investigations. The likelihood us attracting gigantic traffic to in our first five years is probably low, so we’re going to want to extend our reach through partnerships.

In general, in our investigations, we often reach out to academic and subject matter experts for advice. We’re not actually statisticians. We know we are just amateurs. We always reach out to experts on a case by case basis for advice on investigations. We may formulate that into an advisory group, but we haven’t decided yet. Of course people can donate! I know we did receive an enormous gift and we are so grateful, but of course if we want to run this — that’s about four years of funding, so they can contribute to the fifth year!

[Here’s that interlude with Sue Gardner, The Markup’s cofounder and executive director, with more on the finances.]

Gardner: The story of the societal effects on technology is remarkably undercovered. We have a lot of tech coverage — I’ve been living in San Francisco through a lot of it. A lot of it has been gossipy stuff or the exciting rollouts of new products and services, or it’s been the business coverage and effect with stock prices. There has been remarkably little coverage of the societal effects of new technologies, and it is the story of our time. We felt that there wasn’t yet — until now — a journalistic organization that focused solely on the societal effects of technology. It was a big screaming gap in the media landscape.

Schmidt: What are the priorities of the $23 million The Markup has raised going forward in these first few years? How do you see the financial model building out around that?

Gardner: Right, the first couple of years we’re aiming to do two things: We need to build the news organization. Jeff and Julia at ProPublica pioneered the practices of bringing data science to journalism so we’re going to try to scale up the model from ProPublica and institutionalize it.

The other thing we’re going to be working on is trying to find a sustainable model for that kind of journalism. Investigative journalism is the most expensive kind of journalism. Data-centered investigative journalism is even more expensive. It’s a niche product. It’s not broadly appealing to large numbers of people. Journalism is best when it’s paid for by the users. Then all the incentives are virtuously aligned.

What we plan to do is exactly what I did at the Wikimedia Foundation. When I went there, we did not have a business model. We were a nonprofit, but we weren’t bringing in a lot of money. My first job was to figure out sustainability for Wikipedia. When you look at it now, it seems really obvious for what the model for Wikipedia should be, but it was not obvious in 2007. We deliberately set out to spend two years experimenting with different revenue model. We solicited major gifts, we spoke with foundations and got grants for the organization, we did what we called the ‘many small donors’ model, and we experimented with various kinds of earned income. I had always hoped when we started that the ‘many small donors’ model would be successful, and it did turn out to be successful.

I want to do the same thing with The Markup. We’re going to experiment for probably around two years and we’re going to double down on what seems to be working. That is what Craig Newmark’s money and the major grantmakers’ money has bought us — that runway, so we have a couple years to experiment and we have some time to figure out what will work in the longterm.

Schmidt: What are some of the experiments you’re eyeing?

Gardner: We’re going to need to experiment with appealing to people beyond those who read and consume the news products. Investigative journalism, in particular, is very niche and the audiences for it are very small. If you approach it as a purely consumer product, you limit how much money you can raise. I think it’s a mistake to think of journalism as purely a consumer product — it is a consumer product, but there’s also an argument to be made that journalism is also a public good. I benefit, as a person in the world, from the work that the International Consortium of Investigative Journalists or the Organized Crime and Corruption Reporting Project is doing. I benefit even if I don’t consume the stories directly, because journalism has a role in holding power to account, which is separate from its role in creating an informed society. One of the messages we’re going to be experimenting with is an argument that it’s a public good and that the public wants the tech industry and institutional users of technology to be held to account independent of whether they read our work or not. It’s a public good and it should be supported as much as being a consumer product.

[Okay, back to Julia.]

Angwin: We are not going to experiment with advertising. We’ve ruled that out. We are not going to be taking corporate money. We don’t take government money. I hope philanthropy can support investigative journalism for years to come, but it would be wise to look at other options as well.

I really, really am excited to try to build a model for a new way for doing tech-driven journalism. There was a time when journalists really didn’t know anything about business, and then there was the effort to educate journalists about finance and I was part of that. I got the Knight-Bagehot fellowship at Columbia and I ended up getting my MBA because I was a business journalist and I wanted to have that expertise in the area I covered. I feel like we need that in technology.

Technology is invading every part of our lives, and it is used as a cover for political decisions. Journalists in every field need to have more skills to investigate those types of decision-making that is embedded in technology. I’m hoping we build a model that is replicated.

I see us as a FiveThirtyEight — when FiveThirtyEight started, they were the first ones doing major meta statistical analysis of polls and using that to inform political coverage. Then everybody copied it, like with The Upshot. Part of their success was extended through invitation. I hope that happens to us. I still want us to exist, I don’t want to be copied out of existence, but if we can build a model about how you can do this kind of work that’s journalists paired with technologists and expertise — I would be thrilled if that spread to other newsrooms and became a practice and a field.

Image of code markup from Markus Spiske used under a Creative Commons license.

POSTED     Sept. 24, 2018, 2:55 p.m.
SEE MORE ON Reporting & Production
Show tags
Join the 60,000 who get the freshest future-of-journalism news in our daily email.
Are you willing to pay for Prepare to be asked before year’s end
The cable news network plans to launch a new subscription product — details TBD — by the end of 2024. Will Mark Thompson repeat his New York Times success, or is CNN too different a brand to get people spending?
Errol Morris on whether you should be afraid of generative AI in documentaries
“Our task is to get back to the real world, to the extent that it is recoverable.”
In the world’s tech capital, Gazetteer SF is staying off platforms to produce good local journalism
“Thank goodness that the mandate will never be to look what’s getting the most Twitter likes.”