Nieman Foundation at Harvard
What’s with the rise of “fact-based journalism”?
ABOUT                    SUBSCRIBE
Feb. 1, 2019, 8:45 a.m.
Audience & Social

Individually, people aren’t great at judging news sources. En masse, they’re almost the same as professional fact-checkers

Plus: 2018’s most popular health articles were plagued by misinformation, and one of Facebook’s fact-checking partners shares details on how much it’s paid.

The growing stream of reporting on and data about fake news, misinformation, partisan content, and news literacy is hard to keep up with. This weekly roundup offers the highlights of what you might have missed.

“We find remarkably high agreement between fact-checkers and laypeople.” Building on a draft paper from last year, psychologists Gordon Pennycook and David Rand have a new study showing that people across the political spectrum rate mainstream news sources as more trustworthy than hyperpartisan and fake news sites — and that “politically balanced layperson ratings were strongly correlated with ratings provided by professional fact-checkers.” Herein lies a possible solution for social media companies trying to decide which news content to up-rank: Maybe they could try trusting the crowd. “Incorporating the trust ratings of laypeople into social media ranking algorithms may prove an effective intervention against misinformation, fake news, and news content with heavy political bias,” the authors write.

Pennycook and Rand did find “clear partisan differences in trust of mainstream news.” In their studies of about 2,000 people, Democrats were much more likely to trust mainstream media outlets than Republicans were. (The one unsurprising exception was Fox News, if indeed you deem it a mainstream media outlet, which Republicans trust more.) But “both Democrats and Republicans gave mainstream media sources substantially higher scores than either hyperpartisan sites or fake news sites.” Combine them all and they do a good job of judging news:

When calculating an overall trust rating for each outlet by applying equal weights to Democrats and Republicans (creating a “politically balanced” layperson rating that should not be susceptible to critiques of liberal bias), every single mainstream media outlet received a higher score than every single hyperpartisan or fake news site (with the exception of “” in study 1). This remains true when restricting only to the most ideological participants in our sample, when considering only men versus women, and across different age ranges. We also note that a nationally representative weighting would place slightly more weight on the ratings of Democrats than Republicans and, thus, perform even better than the politically balanced rating we focus on here.

The study also includes the ratings of eight professional fact-checkers. Like the laypeople surveyed, the professionals “rated mainstream outlets as significantly more trustworthy than either hyperpartisan sites or fake news sites.”

Ultimately, “we find remarkably high agreement between fact-checkers and laypeople,” the authors write, though they note that “this agreement is largely driven by both laypeople and fact-checkers given very low ratings to hyperpartisan and fake news sites” — there is much more disagreement between the groups when they’re only asked to rate mainstream news sources.

“Incorporating the trust ratings of laypeople into social media ranking algorithms may effectively identify low-quality news outlets and could well reduce the amount of misinformation circulating online,” the authors write.

There is a caveat that may not surprise you: This study provides yet more evidence that Democrats and Republicans have very different levels of trust in the media.

We found consistent evidence that Democrat individuals were better at assessing the trustworthiness of media outlets than Republican individuals — Democrats showed bigger differences between mainstream and hyperpartisan or fake outlets and, consequentially, their ratings were more strongly correlated with those of professional fact-checkers. Importantly, these differences are due to more than just alignment between participants’ partisanship and sources’ political slant (e.g., the perception that most mainstream sources are left-leaning)…

Furthermore, these differences were not primarily due to differences in attitudes toward unreliable outlets —most participants agreed that hyperpartisan and fake news sites were untrustworthy. Instead, Republicans were substantially more distrusting of mainstream outlets compared with Democrats.

7 out of 10 of the most popular health articles of 2018 included at least some misinformation. Health Feedback is a nonpartisan, worldwide coalition of scientists fact-checking health and medical news. Working with the Credibility Coalition, it assessed the credibility of the most popular health articles of 2018.

Out of 10 articles, scientists rated only three rated as being credible or very credible — and all three were published by Time Magazine (here, here, and here). Of the seven remaining articles, four received “neutral” ratings, and three were rated very low. Of those three, two “were published in websites of dubious origin” (the sites: urhealthguide and Reporting The Truth), but one was a book excerpt published by The Guardian.

I know that you’re right, but I don’t care. What do people do when their favored presidential candidate gets fact-checked? They do not change their minds, according to a new study in Political Behavior from Brendan Nyhan, Ethan Porter, Jason Reifler, and Thomas Wood. They conducted two studies of voters during the 2016 election, fact-checking Trump’s claims about crime and unemployment. They found that “people express more factually accurate beliefs after exposure to factchecks. These effects hold even when factchecks target their preferred candidate.” But “we find no evidence that changes in factual beliefs in a claim made by a candidate affect voter preferences during a presidential election.”

France’s Libération shares details about its fact-checking partnership with Facebook. Libération is one of several fact-checking partners working with Facebook in France. Alexios Mantzarlis applauded its article about how the partnership works and how much money it makes.

Illustration from L.M. Glackens’ The Yellow Press (1910) via The Public Domain Review.

Laura Hazard Owen is the editor of Nieman Lab. You can reach her via email ( or Twitter DM (@laurahazardowen).
POSTED     Feb. 1, 2019, 8:45 a.m.
SEE MORE ON Audience & Social
PART OF A SERIES     Real News About Fake News
Show tags
Join the 60,000 who get the freshest future-of-journalism news in our daily email.
What’s with the rise of “fact-based journalism”?
“To describe one form of journalism as ‘fact-based’ is to tacitly acknowledge that there is also such a thing as ‘non-fact-based journalism.’ And there isn’t.”
Britney Spears and the generational shift in celebrity coverage
“There was just this nastiness that emerged in the way celebrities were covered in the 2000s.”
How to b-e-e of use: Signal Cleveland hosts second annual community spelling contest
“Listening is great, and talking to community members is great, but we also have to figure out how to be of use.”