Twitter  Why Atlantic Media is funding a new social platform called This that only allows users to share links once a day  
Nieman Journalism Lab
Pushing to the future of journalism — A project of the Nieman Foundation at Harvard

How do you tell when the news is biased? It depends on how you see yourself

Does the quest for balance in news stories open journalists up to claims of bias? It’s all about the framing.

Take a moment with the headlines from this screenshot of The New York Times homepage from January. Really — it’s a little experiment. Click the image above for a larger view if you need to.


How did you feel about these headlines? Does it matter to you to learn that they actually came from Fox News on the same day? (Screenshot for proof.) This faux home page was created by Dan Schultz, the MIT grad student also responsible for Truth Goggles, using his NewsJack point-and-click “remixer.”

Knowing what you know now, do these headlines seem different to you? If so, you’ve just proved that we detect and judge bias based on things other than what journalists actually write.

This effect has been noticed before. At the University of Michigan, William Youmans and Katie Brown showed the same Al Jazeera English news clip to American audiences, but with a catch: Half saw the news with its original Al Jazeera logo intact, and half saw the same video with a CNN logo instead. Viewers who saw the story with the original Al Jazeera logo rated Al Jazeera as more biased than before they had seen the clip. But people who watched the same footage with the fake CNN logo on it rated CNN as less biased than before!

Does this mean that we judge “bias” by brand, not content? Many people have tried to define what media bias is, and attempted to measure it, but I want to try to answer a different question here: not how we can decide if the news is biased, but how each of us actually does decide — and what it means for journalists.

The hostile media effect

During the Lebanese civil war in 1982, Christian militias in Beirut massacred thousands of Palestinian refugees while Israeli solders stood by. In 1985, researchers showed television news coverage of the event to pro-Israeli and pro-Arab viewers. Both sides thought the coverage was biased against them.

This effect — where both sides feel that a neutral story is biased against them — has been replicated so many times, in so many different cultural settings, with so many types of media and stories, that it has its own name: hostile media effect. The same story can make everyone on all sides think the media is attacking them.

Like a lot of experimental psychological research, the hostile media effect suggests we’re not as smart as we think we are. We might like to think of ourselves as impartial judges of credibility and fairness, but the evidence says otherwise. Liberals and conservatives can (and often do) believe the same news report is biased against both their views; they aren’t both right.

But why does this happen? Specifically, why does it happen for some stories and topics and not others? Discussion of climate change often provokes charges of bias, but discussion of other hugely significant science stories, such as the claimed link between vaccination and autism, usually produces a much smaller outcry.

You see bias when you see yourself as part of a group

Communications researcher Scott Reid has proposed that we can explain the hostile media effect through the psychological theory of self-categorization. This is a theory about personal identity and group identity, and it says that we “self-stereotype,” placing conceptual labels on ourselves just as we might make assumptions about other people. We all have multiple identities of this kind: gender, age, political preferences, race, nationality, subculture, and so on.

To test this, he performed a series of recently published experiments with American students. In the first, he used a survey to ask people whether they thought the media was biased, as well as their personal political orientations, both on a numerical scale from liberal to conservative. The catch was different groups got different cover pages with different sets of instructions. The first set of instructions was neutral:

The purpose of this questionnaire is to get your views of the news media in general.

The second set of instructions was designed to play up feelings of partisanship:

In recent times the differences between Republicans and Democrats have become highly polarized. Many of the issues discussed in the media are seen very differently by Republicans and Democrats. In this context, it is important to gauge people’s views of the media.

The third set of instructions was also designed to reinforce an identity, but in this case an identity that might be common to both liberals and conservatives — that of being an “American” versus the rest of the world.

With increasing globalization, it has become apparent that the media differs across countries and cultures. Al Jazeera has become the voice for much of the Arab world, both within the United States and in the Middle East. Given these changes, it is important to gauge people’s views of the news media in the United States.

And, oddly enough, the same survey gave different results, depending on the instructions:

Each of the lines on this graph shows how people’s perception of bias varied with their political orientation. The downward slope means that the more conservative someone was — the farther to the right on the “political position” scale — the more they perceived the media as hostile to Republicans, just as expected.

The surprising thing is that the strength of this perception depended on the framing each group had been given. When people were prompted to think about Republicans and Democrats, they perceived more media bias against their views, as indicated by the steep dashed line. When they were instructed to think about America vs. the world, they perceived slightly less bias than the neutral condition, as indicated by the shallow dotted line. Our perception of bias changes depending on the self-identity we currently have in mind.

This self-categorization explanation also predicts that people who are more partisan perceive greater bias, even when the news is in their favor. In Reid’s second experiment, people read an article about polling numbers for the 2008 presidential primaries, containing language like “among Republicans, former New York mayor Rudy Giuliani maintained a 14-point lead over Arizona Sen. John McCain for the Republican presidential nomination,” and similar statements about the Democratic candidates. This time, the source of the information was manipulated: One group saw the poll attributed to the “Economic Policy Institute, a Democrat think tank and polling agency,” while the other was told it came from the “American Enterprise Institute, a Republican think tank and polling agency.”

In this purely factual scenario — dry-as-toast poll numbers, no opinions, no editorializing — respondents still had completely different reactions depending on the source. As you might expect, people who believed that the poll numbers came from the American Enterprise Institute thought that the story was biased towards Giuliani (and vice versa), confirming the hostile media effect. But the perception of favoritism increased not according to whether the reader personally identified as Republican or Democratic, but on how strong this identification was. The implication is that if you feel strongly about your group, you’re likely to see all news as more biased — even when the bias favors you.

Reid’s final experiment tested perceptions of overt attacks. He used a scathing review of Michael Moore’s Fahrenheit 911, originally published on Slate, which begins:

One of the many problems with the American left, and indeed of the American left, has been its image and self-image as something rather too solemn, mirthless, herbivorous, dull, monochrome, righteous, and boring.

The copy given to subjects (falsely) claimed the author was a member of either a Democrat or a Republican think tank. (In reality, the author was the late Christopher Hitchens.) As you might expect, people who identified as Republicans saw the review as more neutral, regardless of who they thought wrote it. The strange thing is that strong Democrats actually saw the review as slightly in favor of Democrats when they believed it was written by a Democrat! We interpret criticism completely differently depending on how we see the relationship between ourselves and the author.

What’s a journalist to do?

The first defense against accusations of bias is to report fairly. But the hostile media effect pretty much guarantees that some stories are going to be hated by just about everyone, no matter how they’re written. I suppose this is no surprise for any journalist who reads the comments section, but it has implications for how news organizations might respond to such accusations.

This research also suggests that the longstanding practice of journalists hiding their personal affiliations might actually be effective at reducing perceived bias. But only up to a point: To avoid charges of bias, the audience needs to be able to see the journalist as fundamentally one of them. This might require getting closer to the audience, not hiding from them. If we each live inside of many identities, then there are many possible ways to connect; conversely, it would be helpful to know, empirically, under what conditions a journalist’s politics are actually going to be a problem for readers, and for which readers.

We might also want to consider our framing more carefully. Because perceptions of bias depend on how we are thinking about our identity in that moment, if we can find a way to tell our stories outside of partisan frames, we might also reduce feelings of unfairness. The trick would be to shy away from invoking divisive identities, preferring frames that allow members of a polarized audience to see themselves as part of the same group. (In this regard, the classic “balanced” article that quotes starkly opposing sides might be a particularly bad choice.)

Encouraging the audience to perceive itself as unified — this seems simplistic, or naïve. But the consideration of identity is foundational to fields like mediation and conflict resolution. Experimental evidence suggests that it might be important in journalism too.

What to read next
Ken Doctor    Aug. 13, 2014
If newspapers are going to have to survive on their own, the first numbers aren’t encouraging. In southern California, we could see big movement fast.
  • http://môniyâ môniyâw linguist

    Hallo! Nice article. This is something those of us in a (slightly) different world also think about, though differently from the MIT/PoliSci/Journalism way you’re looking at. Multiple points of view are good, and I enjoyed yours.

    In sociolinguistics and ethnology, people like me would say that the Newspaper is a framing device – a genre – that helps people interpret the messages within it. This is much like the linguistic cues we see when we read something like the following: 
    “The villagers on that island believe strange things. The sky is made of jello. There are carnivorous pigs in the ocean.” 

    The first sentence is a framing device on subsequent speech, and tells you whose point of view you’re interpreting. (i.e. the villagers on the island and not the speaker’s)

    The aboriginal language group I work with will do this explicitly, so that you know whose point of view you’re representing. It’s a way to keep chains of information and perspectives clear – especially powerful in an oral culture.

    If you’re interested, have a look at work on Modal Subordination, stuff by Dell Hymes, Gregory Bateson, Richard Bauman, etc.

  • Möp_möp

    thanks for the article. its always funny to read “the conservatives control the media” on liberal sites and “the liberals control the media” on conservative sites.

  • Former_scribe

    As a working journalist, I judged my efforts at objectivity a success if both sides of an issue were equally mad at me. If they were all equally pissed, I was probably driving pretty straight down the middle. I suspect the same test still applies, and perhaps even the same mindset, at least among journalists who consider themselves more observers than advocates. (Once there was no other kind of journalist except the Observer.)

  • Möp_möp

     its a good judge, as soon as i read a comment that talks about “a good article, very well balanced” i get very suspicious because in 9 from 10 cases such a comment is found below a very one sided article.

  • Alex M-L

    When they were instructed to think about America vs. the world, they perceived slightly less bias then the neutral condition, as indicated by the shallow dotted line. 
    ***than the neutral condition

  • Joshua Benton

    Thanks — fixed.

  • Geochief

    My problem with most media is how shallow it is, more emphasis is placed on being the first to report something than on doing it well. TV news is the worst, seldom is any historical context given to what is happening, never any background on the protagonists. The real bias is in the perception that we all have grown up with, of how we got to certain points of view, are we spending more under the current administration than in the past? Are there suddenly new and radical funding of social programs?
    Are income taxes really more than in prior years? There are facts that either support or refute most of these claims but the news NEVER bothers to try and point that out. Oh and I don’t care what administration you want to talk about, I’m an independent. The only two guys who even approach reliable journalism are Jon Stewart and Steven Colbert and they will no doubt remind you that they are comedians! The rest of the lot has sunken to sensationalist garbage or celebrity gossip.

    You want to do some great journalism start with this simple story, how do you work for 40 years, saving 10% of your income each year and then retire and get paid 60% of your preretirement income  for 30 years? Or social security or medicare or really try and explain the federal budget from year to year what went up what went down.

  • Mark

     So you never bothered to examine your own product? That “both sides criticize me, so I can’t legitimately be criticized” is one of the weakest of all defenses. Obviously, it is entirely possible that one side is actually right, and the other wrong. But that never occurred to you?

  • Mark

    What I learned today: we in the media are incapable of bias. Bias only exists in the minds of those stupid readers. 

  • Verity

     I must agree. This article was incredibly self-serving. The vast majority of the mainstream national media self identifies as Democrats. This is also the case for Hollywood, most university professors (and goodness does this include Law Professors). In fact, this is pretty easy to see based on campaign contributions issued by a large majority in these groups. It’s not even close as they give large chucks to the Democratic Party.

    I enjoy reading the NY Times, Washington Post, and listening to NPR, but I do realize that at the very least, I’m getting a left of center view of things. This is fine by me as I know what I”m getting and make sure to try to read from varied sources.

    I’ve always found it odd that though a large percentage of the media self identifies as Democrats (and this is pretty obvious if one consumes any of this media), we are all supposed to pretend that it’s all in our heads.

    Just as I would not pretend that Fox News is an objective news source, I also wouldn’t pretend that NBC/MSNBC and the NY Times are objective news sources either. This isn’t because I have any axe to grind (I’ve only ever been registered as a Democrat) it’s because that is the way things are.

  • Ross Williams

    Both sides thought the coverage was biased against them.
    That’s probably because they are both right. The media has its own bias and it often misrepresents other points of view in order to create its own narrative. You can see that here:
    “This effect — where both sides feel that a neutral story is biased against them”
    Where is the evidence that the story in question was “neutral”? There isn’t any. In fact, apparently everyone else agreed the story wasn’t neutral. This story portrays the problem as flaw in peoples perceptions called “hostile media effect”, but there is no evidence that people’s perception of the media is inaccurate. 

    The media is largely concerned with building and holding the interest of an audience. The information they choose to provide is all filtered through that bias. It means creating simple stories with comfortable, easily digested narratives, regardless of how complex or uncomfortable the story is. It means choosing the interesting facts whether the are representative or provide an accurate understanding.    

  • ben knight

    What horseshit. Bias is bias, it isn’t relative. 

  • LisaP

    I saw that it was the and I was expecting that I would consider it biased. Then I read the headlines themselves, noticing that the one biased piece was labeled opinion, and I thought….”hmmm, maybe I misjudged them”…
    So I guess that means I’m pretty unbiased….or rather, I’m self-aware of my biases. 
    As a human, that’s the most you can strive for, right?

  • LisaP

    *the New York Times

  • JackP, Hobbs, NM

    Nowhere in this article do I see any admission that the media is ever biased. I find this strange in that the journalist that directed me to this page has a standard reply for claims that the Main Stream Media is leaning/slanted/biased towards the left; “There is also a right-wing media.” which would mean that there is indeed bias in the media. Given that I am, both socially and fiscally, a conservative, I try to adjust my view to account for my bias before I make a determination on whether or not the story being presented shows bias. Admittedly, I am not always successful, and like LisaP, the pieces labeled as opinion, I believe, deserve some extra latitude. Reading the same basic story from multiple sources, both from those *I* consider to be left-leaning and right-leaning, does seem to help in getting closer to the actual truth.

  • Lobo

    Garbage…. A real example would be CNN had one of their anchors on the air just the other night  saying directly into the screen that Romney has lied about everything from tax cuts to Bain Capital. Not one mention on the inexperience of Obama or his questionable past pre-president. An not Romney to defend himself. I voted for Obama in 2008 and even I can see this, the U.S. media needs to have some dignity and respect for the American People!

  • Jonathan Ward

    I just wrote a blog entry on this topic, from a non journalists point of view. You brought up some good points and gave me some ideas to research.

  • Warbo

    How do you tell when the news is biased? When they’re reporting it. All news sources are biased. Anyone who says differently is selling their preferred network.