Nieman Foundation at Harvard
A journalistic lesson for an algorithmic age: Let the scientific method be your guide
ABOUT                    SUBSCRIBE
Jan. 5, 2023, 9:10 a.m.
Reporting & Production

Partisan media offers easier-to-read political news than mainstream outlets, study suggests

Hyperpartisan outlets use words a larger share of the population understands, researchers found.

When researchers analyzed almost 6,000 political news stories produced by partisan and nonpartisan media outlets in 2021, three things became clear:

  • Media outlets with extreme biases — regardless of whether it was a conservative or liberal bias — tended to use shorter sentences and less formal language than nonpartisan outlets.
  • Mainstream news organizations, as a whole, wrote at a higher reading level. For example, stories from Reuters wire service were written at the level of someone who had completed a year and a half of college, on average. Meanwhile, Wonkette, a far-left online publication, wrote at a ninth-grade reading level, on average. American Thinker, a far-right online publication, wrote at an eleventh-grade level.
  • Far-right and far-left outlets took a more negative tone than nonpartisan outlets. They generally had a lower ratio of positive to negative words.

The researchers describe their findings in a paper forthcoming in Journalism Studies, “At the Extremes: Assessing Readability, Grade Level, Sentiment, and Tone in US Media Outlets.” While the findings apply only to the 20 media outlets studied, they provide insights into how these news organizations differ and why some audiences seek information from partisan media, which often don’t adhere to professional journalism ethics or norms.

The paper’s lead author, journalist-turned-researcher Jessica F. Sparks, says people generally prefer simpler, easier-to-understand language. “Hyperpartisan” outlets use words a larger share of the population understands. They also present issues and events in simpler terms — often by leaving out key facts and context, especially when it benefits one political party over another, Sparks points out.

“We, as humans, are what we call ‘cognitive misers,'” she explains. “There are people who like to use more cognitive energy, but most of us don’t. And so, when we think about it that way, when we read simplified text — especially text that frames things as us-versus-them — it’s easier to process that than [news reports that examine] the complex goings-on of Washington.”

Sparks, a doctoral student at the University of Florida, and coauthor Jay Hmielowski, an associate professor at UF’s College of Journalism and Communications, analyzed 5,847 news stories published during three one-week periods in early 2021. They only looked at stories that did not appear behind paywalls, however, partly because they wanted to focus on coverage accessible to anyone.

They examined political coverage from a range of news organizations, including broadcast, digital-native and news wire outlets. Of the 20 studied, seven were nonpartisan. Three were far-left outlets, two were far-right and eight outlets produced stories that tended to lean at least somewhat to the right or left.

Sparks and Hmielowski determined outlet bias based on information gathered from three news bias-rating websites: AllSides, Ad Fontes Media and Media Bias/Fact Check. They gauged the negativity of each outlet’s coverage with Readable, a tool that compares the number of negative words — “bashing” and “forbidden,” for example — to positive words — for instance, “accomplished” and “great” — in a written work. Readable assessed stories’ formality based on factors such as sentence structure, the prevalence of impersonal, formal language and whether personal pronouns such as “we” and “I” appeared in the story.

The researchers relied on two widely used readability scales, the Flesch-Kincaid Grade Level and the Gunning Fog Index, to determine the grade level needed to understand news stories produced by each outlet.

Sparks says she was surprised to find similarities between the far-left and far-right media organizations she studied. Both wrote at a lower grade level and used simpler, less formal language.

“Most research has suggested that right-wing media is very distinct from other forms of media and what we found is that’s not necessarily the case with readability,” she adds.

Advice for journalists

It’s unclear whether partisan outlets intentionally make their content easier to read and understand. But Sparks and Hmielowski indicate their readability helps set them apart from nonpartisan news organizations.

“If audiences are seeking content that reflects their attitudes and that rejects mainstream journalism, partisan media outlets on both sides of the political spectrum benefit from differentiating themselves both in content substance and content style,” the researchers write. “Sentence structure, informality, and tone might be one way to achieve that.”

Sparks, whose journalism has appeared in publications such as The Washington Post and Indianapolis Monthly, says it’s important for journalists working in nonpartisan newsrooms to use simple language. But they also need to provide the public with a nuanced understanding of the issue or event they are reporting on, even if it complicates a news story.

Her advice to reporters: Take time to explain the various sides of an issue. If partisans are oversimplifying an issue, note that in your coverage.

“This study, I think it gives us a better idea of what we’re up against and what journalists at reputable organizations might want to consider when they’re writing news about heavily partisan politics,” she says.

Denise-Marie Ordway is managing editor at The Journalist’s Resource at the Harvard Kennedy School’s Shorenstein Center, where this piece was originally published.

Photo by Joshua Hoehne on Unsplash.

POSTED     Jan. 5, 2023, 9:10 a.m.
SEE MORE ON Reporting & Production
Show tags
Join the 60,000 who get the freshest future-of-journalism news in our daily email.
A journalistic lesson for an algorithmic age: Let the scientific method be your guide
“One of the best parts about using the scientific method as a guide is that it moves us beyond the endless debates about whether journalism is ‘fair’ or ‘objective.’ Rather than focus on fairness, it’s better to focus on what you know and what you don’t know.”
The future of local news is “civic information,” not “declining legacy systems,” says new report
“In this vision, the community librarian facilitating conversations around authoritative, trusted digital news is as celebrated as the dogged reporter pursuing a scoop.”
Is text-generating AI an industry killer or just another wave of hype?
“There can potentially be massive shifts, benefits, and risks in many industries, but I cannot see a scenario where this is a ‘sky is falling’ kind of issue.”