Nieman Foundation at Harvard
BREAKING: The ways people hear about big news these days; “into a million pieces,” says source
ABOUT                    SUBSCRIBE
March 5, 2018, 3:11 p.m.
Reporting & Production

This new initiative deploys humans to review, research, and rate U.S. news sites

“We’re making the kinds of judgments that we think will be noncontroversial, but still are the kind of judgments that platforms, rightly, don’t want to be making themselves.”

The Denver Post? GREEN. The fake news site the Denver Guardian? RED. A site that’s not putting out deliberately fake news, but is overwhelmingly influenced in its coverage by a funder that it’s not eager to disclose? Maybe a YELLOW.

That’s the proposed ratings system underlying NewsGuard, an initiative launched by Steven Brill (The American Lawyer, Court TV) and L. Gordon Crovitz (Wall Street Journal, Denverite), that intends to hire people with journalism backgrounds to assign more than 7,500 news sources one of three ratings — green, yellow, or red — and provide a 200- to 300-word write-up on each source’s funding, its coverage, its potential special interests, and how it fits in with the rest of the news ecosystem. (You can read a few sample write-ups, which NewsGuard is calling “nutrition labels,” on its site here.) Not all of the sites in a given color category are equal, the founders acknowledge (“some are much more fair and accurate in their reporting than others”), but they hope the nutrition labels will clarify the reasoning for each rating, and dive into additional nuances such as the differences between opinion and news sections.

“We’re not presuming we can solve all the problems or that we’re the world’s content umpires. But what we’ve told the platforms that asked us — and this has seemed to be convincing to them — is that we have one solution, and if a competitor were to propose to do the same thing with their people, you should license us both, and let your users decide which is transparent and more reliable,” Brill said. “The overall point here is that we’re trying to show people here’s a difference between the Denver Guardian and The Denver Post, and there’s a difference between the Washington Post writing about the pros and cons of fracking versus writing about fracking.”

Brill and Crovitz are looking to hire dozens of people with journalism backgrounds for the venture, which is based in New York and Chicago, and they intend to license NewsGuard’s encyclopedia of news sources to social media platforms and search engines. (Platforms licensing the database can, for instance, choose to surface only the nutrition labels or only the green-yellow-red designations for users, or choose to only make visible nutrition labels but use ratings as a ranking signal.) Another potential revenue stream could come from licensing the database NewsGuard builds up to advertisers who want to be spared any embarrassment that comes from advertising on deliberately fake sites, by offering them whitelists of safer — say, not red-rated — news sites.

Its main pool of sites, the founders and co-CEOs have said, “account for 98 percent of the news articles read and shared in the English language online in the U.S.” NewsGuard is working with a mix of third-party measurement organizations that look at engagement on social media and other platforms and direct web traffic to come up with this main pool of sites, Brill and Crovitz told me. The news sites include a wide range of news and opinion, entertainment (so yes, TMZ), sports, most local newspaper sites, and niche or trade publications. (They’re in the process of ironing out contracts and intend to disclose those third-party organizations as well.)

The effort is backed by $6 million in funding so far and is supported by foundations like Knight, the advertising and PR agency Publicis Groupe, as well as individuals (the list of investors is available here. Disclosure: Knight also supports Nieman Lab.)

Two NewsGuard “analysts” research each news source and then settle on a rating (or not, if they feel they don’t have enough information to be confident in a rating). If the raters disagree, NewsGuard’s executive editor Jim Warren, managing editor Eric Effron, and its editor-in-chief will weigh in, and NewsGuard will make available all the backgrounds and conflicts and thought processes of every person involved in the decision process (it’s looking to add a second layer of fact-checkers who will fact-check the nutrition labels analysts have written. The NewsGuard site lists criteria that feed into each rating, such as points for whether a news site participates in the Trust Project, or whether a site appears to be trying to obscure its funders or its ideological bent. NewsGuard will publish corrections and comments, and welcomes suggestions to rate any sites it’s missed, Brill and Crovitz said.

I asked what ratings some of the news sites that have sample nutrition labels posted (, are getting; Brill and Crovitz told me they are hesitant to explain ratings before these sites have gone through the formalized ratings process. But based on reading their ratings criteria and its nutrition label, a site like, funded by the American Petroleum Institute, would likely end up a YELLOW.

“We’re making the kinds of judgments that we think will be noncontroversial, but still are the kind of judgments that platforms, rightly, don’t want to be making themselves. They don’t want to put their finger on the scale at all,” Crovitz said. “A use case is…maybe that teenager who’s trying to figure out where a source of news comes from. If your crazy uncle sends you a news story that looks kind of crazy, it’d be useful for you to know whether that actually comes from a fake news site.” As NewsGuard builds up its database of ratings and nutrition labels, it wants to offer a browser plug-in version of its ratings and nutrition labels for free for educational purposes, such as to media literacy groups.

Part of its efforts also include making available a 27-7 “SWAT team” that responds to breaking news and news items that are suddenly trending — their way of addressing the recent swell of criticisms that platforms and their algorithms seem to be unable to detect what individuals can easily spot.

I told Brill and Crovitz that I personally didn’t feel confident about a normal news consumer’s willingness to read and accept all these ratings and nutrition labels presented in this way. But platforms like Facebook have rolled out some product tests that suggest their thinking aligns with what Brill and Crovitz want to do with NewsGuard, such as adding a “more context” button on a news article shared to the News Feed that pulls from sources like the publisher’s Wikipedia page.

“I don’t think what we’re doing is analogous to any of the efforts out there already,” Brill said, and highlighted NewsGuard’s collaboration with the Trust Project. “What we’re doing is achieving scale by rating sites, while not getting bogged down in the nuances of individual articles. We will use, for instance, the information that fact-checking groups come up with to inform our nutrition labels and ratings. If there’s a site that frequently runs afoul of fact-checking groups, we look at that, we’ll evaluate if these issues are legitimate, and that will make its way into our nutrition label. We’re doing something that is at once less dramatic and controversial, and at the same time, we think will be more effective and sweeping.”

Photo of traffic lights, by Dave’s Domain. Used under a Creative Commons license.

POSTED     March 5, 2018, 3:11 p.m.
SEE MORE ON Reporting & Production
Show tags
Join the 60,000 who get the freshest future-of-journalism news in our daily email.
BREAKING: The ways people hear about big news these days; “into a million pieces,” says source
The New York Times and the Washington Post compete with meme accounts for the chance to be first with a big headline.
In 1924, a magazine ran a contest: “Who is to pay for broadcasting and how?” A century later, we’re still asking the same question
Radio Broadcast received close to a thousand entries to its contest — but ultimately rejected them all.
You’re more likely to believe fake news shared by someone you barely know than by your best friend
“The strength of weak ties” applies to misinformation, too.