A few months ago, a group of researchers was interested in finding out whether or not Americans supported what they considered five “core journalism values.” So they surveyed 2,727 Americans and asked whether they agreed or disagreed with 20 different statements. This was one of them:

We need to put a spotlight on problems in society in order to solve them.

And this is the breakdown of how people responded:

Strongly agree: 27%
Moderately agree: 27%
Slightly agree: 18%

Slightly disagree: 4%
Moderately disagree: 1%
Strongly disagree: 1%

Add those up and you get 72% of people agreeing with the statement to some degree and 6% disagreeing to some degree. Pretty strong support!

Last week, those researchers released their findings, and they got a fair amount of attention. Here’s how The Washington Post’s Margaret Sullivan summed up this piece of it:

The [journalistic] value drawing the least support is the idea that a good way to make society better is to spotlight its problems. Only about 3 in 10 agree.

Wait a minute: How in the world do you get from 72% agreeing to “only about 3 in 10”?

What about another core journalism value, transparency? Here are two other statements those 2,727 Americans were each asked about:

Transparency is usually the best cure for what’s wrong in the world.

On balance, it’s usually better for the public to know than for things to be kept secret.

For that first statement, 63% of people agreed versus 15% who disagreed to some degree. For the second, 69% agreed against only 9% who disagreed.

And yet the researchers say their work shows transparency is a journalistic value only 44% of Americans support. Huh?

Okay, one more. The researchers consider “giving voice to the less powerful” a core journalism value. So they asked those 2,272 people about this statement:

It’s important to offer a voice to the voiceless.

It was a blowout: 74% of people agreed with the statement, while only 4% disagreed.

So what share of Americans do the researchers say support the journalism value of “giving a voice to the less powerful”? Just half: 50% do, 50% don’t.

What in the world is going on here? Why is a study consistently seeming to reduce the number of Americans who support the values of journalism?

The study in question — which was done by the Media Insight Project, a joint effort of the American Press Institute and the Associated Press-NORC Center for Public Affairs Research — contains a lot of interesting data and analysis, some of which I’ll get to later.

But its top-line finding — summarized by a Post headline writer as “Bad news for journalists: The public doesn’t share our values” — is bogus. Or, at a minimum, unsupported by the methodology in use here. There is no reason to believe, based on this data, that Americans have somehow abandoned the basic values of democratic governance, or that we noble journalists are left to fight the lonely fight for accountability.

And whenever we exaggerate the distance between journalists and their readers — a genre of media reporting I call “mistrust porn” — we both exacerbate the problems that do exist and hand a weapon to people who want to attack mainstream journalism.

Besides the Post (and journalism-specific sites like us and CJR), what news outlets picked up this study? Here are a couple: Fox News and RT, neither one a noted friend to fact-based reporting. (RT: “Are journalists betraying their values? No, it’s the public who are wrong, says Washington Post writer.” Fox: “‘It’s the uncivilized, unwashed masses who don’t share our noble values,’ one critic joked in response to Margaret Sullivan.”) Something called Liberty Nation News said the study “confirmed the media’s sinking reputation,” while James O’Keefe tweeted that it proved the news industry “gaslights the people.” (Right before Twitter suspended him.)

This must be parody.

Author laments public mistrust of press & identifies four “core values” of journalists:

Transparency
Factuality
Spotlighting wrongdoing
Giving a voice to the voiceless

Ignores that the press ONLY FOLLOWS THEM WHEN IT HELPS DEMOCRATS https://t.co/W1781SIkfd

— Ted Cruz (@tedcruz) April 14, 2021

There are two main problems with the claims being made off of this API/AP-NORC study. The first is that the study invents an artificially stringent standard for what it means to “support” a journalism value.

The second is that, while it surveys thousands of civilians to discover their thoughts on various values, it never actually surveys journalists to see how they feel about them. And given the weird standard they’re using, I can assure you lots of journalists would be closer to the general public than whatever platonic form of news righteousness the researchers have in mind.

Let’s start at the beginning. These are the five core journalism values researchers were seeking to measure:

Oversight: This value measures how strongly a person feels there is a need to monitor people in power and know what public officials are doing or saying. The flip side of this value is worry about intrusiveness or this oversight becoming a hindrance, getting in the way, or placing too much importance on insignificant events.

Transparency: This is the idea that transparency is usually the best cure for what’s wrong in the world, and that on balance it’s usually better for things to be public than for things to be kept secret. The inverse is that sometimes the need to keep things secret is more important than the public’s right to know and that most problems can be solved without embarrassing facts being laid out in the open.

Factualism: This value measures whether on balance more facts are always better and facts are the key to knowing what is true. The flip side is that the truth is more than just a matter of adding facts and that this emphasis on factualism can mask bias.

Giving voice to the less powerful: This value measures whether people want to amplify the voices of people who aren’t ordinarily heard and if a society should be judged on how it treats the least fortunate. The inverse instinct is that inequalities will always exist and favoring the least fortunate does not always help them.

Social criticism: This value measures how important people think it is to put a spotlight on a community’s problems in order to solve them. The flip side puts more emphasis on the value of celebrating things that are going right or working well in order to reinforce them and encourage more of them.

How did researchers come up with these core values? They “identified [them] through our own experience,” followed by some brainstorming with a group of journalists. They’re fine — I don’t have a quibble with the values themselves, only how they’re being measured here.

To see how Americans feel about these values, researchers created four statements about each one — 20 in all. For each value, they crafted two statements where agreeing with it indicated that you support the value — that you are pro-transparency, pro-oversight, and so on. The other two statements were worded in such a way that disagreeing with them was the pro-values position.

For example, here are the four statements meant to evaluate people’s support for oversight:

1. The powerful need to be monitored or they will be inclined to abuse their power.

2. It’s important to put some trust in authority figures so they can do their jobs.

3. It’s vital that the public know what government leaders are doing and saying each day.

4. Leaders need to be able to do some things behind closed doors to fulfill their duties.

If you support the value of journalistic oversight, you’re supposed to agree with 1 and 3 and disagree with 2 and 4.

Here are the statements for transparency:

1. On balance, it’s usually better for the public to know than for things to be kept secret.

2. Sometimes the need to keep a secret outweighs the public’s right to know.

3. Transparency is usually the best cure for what’s wrong in the world.

4. Most problems can be addressed without putting embarrassing facts out in the open.

And for giving a voice to the less powerful:

1. A society should be judged by how it treats its least fortunate.

2. Sometimes favoring the least fortunate doesn’t actually help them.

3. It’s important to offer a voice to the voiceless.

4. Inequalities will always exist and you can’t eliminate them.

For these, again, agreeing with 1 and 3 and disagreeing with 2 and 4 are meant to indicate you support the value in question. 2 and 4 are supposed to represent “the antithesis of those values” promoted by 1 and 3.

Do you notice any difference in tone or structure between the questions you’re meant to agree with and the ones you’re not? The affirmative statements — 1 and 3 — tend to be statements of a general civic principle. The powerful need to be monitored. It’s important to offer a voice to the voiceless. It’s vital that the public know what government leaders are doing and saying.

Meanwhile, the negative statements — 2 and 4 — tend to be about limited exceptions to the general civic principles. They’re hardly the “antithesis” of the others. And frankly, they’re the sort of statements that are pretty hard to disagree with if you think about them with any sort of nuance.

I mean, does anyone really think that, no, you should never put any trust into any authority figure, ever? (Only 9% of those surveyed felt that way — the supposed “pro-journalism” position.) Or that no “leaders” should ever be allowed to do anything behind closed doors?

Journalists really do like governmental transparency. But do you know anyone not named Assange who believes the need to keep a secret never outweighs the public’s right to know? I should be able to FOIA the nuclear codes? A list of every woman who’s had an abortion? Every American’s tax returns? If you disagree with “Sometimes the need to keep a secret outweighs the public’s right to know,” you’re saying there should be no secrets, at all, ever.

And how is agreeing that “inequalities will always exist” somehow the “antithesis” of wanting to give voice to the voiceless? “Inequalities will always exist” is, as far as I can tell, a pretty solid lesson to take from 300,000 years of anatomically modern human history. As a factual claim, it doesn’t have anything to do with “Inequalities will always exist…so obviously we shouldn’t even bother to try to fix them,” or “Inequalities will always exist…so would all these poor people please shut the hell up now?”

If you support the journalistic value of factualism, you’re supposed to agree with these two statements:

The more facts people have, the more likely it is they will get to the truth.

For most things, knowing what’s true is a matter of gathering evidence and proof.

Hard to argue with either one, right? They’re basically milquetoast endorsements of rational human thought. And both were overwhelmingly endorsed by those surveyed: 71% agree/7% disagree on the first one, 74% agree/4% disagree on the second.

But you’re also supposed to disagree with these two:

A lot of the time you know enough about something and more facts don’t help.

For a lot of things that matter, facts only get you so far.

I mean, those are also self-evidently true, right?

“No, you can never know enough about anything, you always need more facts — which is why I’ve been googling ‘best microwave oven’ continuously for the past 23 years.”

And if you really think cold, hard facts are all you need in life — that things like emotion, humor, chance, altruism, or love should never distract you from a purely rational response to all of life’s questions — I’m gonna guess you don’t get a lot of second dates, and no, I do not want to read your Substack.

The result of this split of framing is that the “meant-to-agree” statements are easy to agree with — but so are the “meant-to-disagree” statements. And that’s exactly what the data shows: More people agreed than disagreed with all 10 of the affirmative statements. But more people also agreed on 9 of the 10 negative statements.

(I’ll list all the statements and their agree/disagree numbers at the end of this piece.)

Why does that matter? Well, it matters because of the arbitrary standard researchers picked to determine if someone is a “supporter” of a journalism value. To count as a supporter, you must both on average agree with the positive statements and disagree with the negative ones. See the language at the bottom of this chart: “The percent who embrace each value is the percent who on average at least slightly agree with the two affirmative statements and slightly disagree with the two contradictory statements used to measure each value.”

That’s how you get from those seemingly happy numbers I mentioned up top to the seemingly sad ones that follow.

Remember: 12 times as many people agreed with “We need to put a spotlight on problems in society in order to solve them” than disagreed with it. 72% to 6%! And yet some muddled question design and an arbitrary definitional decision turn that into “only about 3 in 10 agree” that “a good way to make society better is to spotlight its problems.”

The report itself fuzzes this up a bit by using different strengths of language when describing what the percentages mean. Sometimes the wording acknowledges that they’re applying a more stringent test of support:

The study finds that not all Americans universally embrace many of the core values that guide journalistic inquiry.

In all, only 11% of Americans unreservedly embrace all five of the journalism principles tested…

Only 11% of Americans fully support all five of the journalism values tested.

But other times, it falls back on just a simple “support” or “endorse,” as did nearly all of the coverage the report got:

Only one of the five core journalism values tested has support of a majority of Americans…

People who most value loyalty and authority are much less likely than others to endorse the idea that there should be a watchdog over those in power.

Fewer endorse the values of oversight, transparency, and social criticism.

C’mon, Josh, you might be saying — you’re being picky. The method they used is probably better at teasing out different attitudes among the people surveyed. And that’s probably true.

The main thrust of the study is using all those answers to cluster survey takers into four groups, based on their responses, and then comparing those groups’ response to various tweaked versions of news stories. The researchers call those clusters The Upholders, The Moralists, The Journalism Supporters, and The Indifferent. While I might not use those names — labeling 20% of the population “The Journalism Supporters” falsely implies the other 80% don’t support journalism — it’s a rich set of data to play with. I thought their cross-measuring of “journalism values” with “moral values” was interesting, and their experimental changing of story structures was informative, if not earth-shattering. And statements that can thin-slice different levels of support can be useful when you’re running a cluster analysis.

The problem comes when you abandon that nuance and just declare people binary supporters/non-supporters of concepts like “transparency.”

Which leads to the other issue here. Remember that Post headline: “Bad news for journalists: The public doesn’t share our values.” But this study doesn’t even try to measure journalists’ values. It pits real-world survey responses against abstract principles — and hey, if there’s a gap between the two, bad news for journalists, right?

I think there’s a real if unconscious hunger for this sort of mistrust porn. You saw it with the post-2016 discourse around “fake news,” which sometimes discounted mistakes made by the press by focusing instead on how irrational and fact-averse some Trump supporters seemed to be. You see it every time someone writes about the decrease in trust in the media since the 1970s — but doesn’t note that “the media” represented a wildly different thing in 1975 than it does today.

(Do you trust “the media”? Is “the media” Walter Cronkite and your straight-arrow local daily newspaper? Or is “the media” a universe that includes both The New York Times and Fox News, NPR and Breitbart, a local daily and a local blogger, earnest investigators and clickbait hucksters? Absolute trust in “the media” in 2021 is logistically impossible, because different parts of “the media” send out contradictory messages every minute of every day.)

Not to mention that, according to Gallup, trust in media actually increased from 32% to 40% between 2016 and 2020, or that trust levels have been just barely below flat for the past 15 years, not rapidly declining. (Trust in the media was actually higher in 2018 than in 2004.)

The traditional journalism business is in a longstanding decline, and there’s a hunger for explanations — and the idea that “the audience is dumb/untrusting/anti-journalism values” might even feel good for a minute. (See, it’s their fault!) And lord knows it can make for a good headline. But it doesn’t help the cause of journalism to make the audience seem more anti-transparency, anti-oversight, or anti-accountability than they actually are.

Update, 4/22/21: In the interest of, well, transparency, API’s Tom Rosenstiel has responded to my criticisms here. I find it unconvincing and am writing a response that I’ll link here when it’s done. But feel free to check out his arguments. Additional update: Here’s my response.

As promised, here are the 20 statements used in the survey and the agree/disagree splits on each. I’ve bolded the response that’s meant to indicate you support the journalism value in question.

Oversight

1. The powerful need to be monitored or they will be inclined to abuse their power. 70% agree, 8% disagree.

2. It’s important to put some trust in authority figures so they can do their jobs. 68% agree, 9% disagree.

3. It’s vital that the public know what government leaders are doing and saying each day. 66% agree, 12% disagree.

4. Leaders need to be able to do some things behind closed doors to fulfill their duties. 50% agree, 27% disagree.

Factualism

1. The more facts people have, the more likely it is they will get to the truth. 71% agree, 7% disagree.

2. A lot of the time you know enough about something and more facts don’t help. 31% agree, 48% disagree. [Note: This was the only statement of the 20 that drew more disagreement than agreement.]

3. For most things, knowing what’s true is a matter of gathering evidence and proof. 74% agree, 4% disagree.

4. For a lot of things that matter, facts only get you so far. 48% agree, 30% disagree.

Giving voice to the less powerful

1. A society should be judged by how it treats its least fortunate. 65% agree, 13% disagree.

2. Sometimes favoring the least fortunate doesn’t actually help them. 52% agree, 26% disagree.

3. It’s important to offer a voice to the voiceless. 74% agree, 4% disagree.

4. Inequalities will always exist and you can’t eliminate them. 56% agree, 23% disagree.

Social criticism

1. We need to put a spotlight on problems in society in order to solve them. 72% agree, 6% disagree.

2. Too much focus on what’s wrong can make things worse. 50% agree, 28% disagree.

3. The way to make a society stronger is through criticizing what’s wrong. 43% agree, 35% disagree.

4. The way to make a society stronger is through celebrating what’s right. 67% agree, 11% disagree.

Transparency

1. On balance, it’s usually better for the public to know than for things to be kept secret. 69% agree, 9% disagree.

2. Sometimes the need to keep a secret outweighs the public’s right to know. 49% agree, 29% disagree.

3. Transparency is usually the best cure for what’s wrong in the world. 63% agree, 15% disagree.

4. Most problems can be addressed without putting embarrassing facts out in the open. 57% agree, 22% disagree.