HOME
          
LATEST STORY
Opening up the archives: JSTOR wants to tie a library to the news
ABOUT                    SUBSCRIBE
March 16, 2012, 12:46 p.m.
A medley of Twitter eggs

Think fast: Is that tweet true or false? How we use credibility cues to make decisions

With little to go on, Twitter users rely a lot on user names and profile images to judge the veracity of tweets.

A medley of Twitter eggs

What makes a tweet credible?

You hear an important person has died, so you turn to Twitter search. Or you see the person’s name is a trending topic and follow the link.

You begin hunting for clues — subconsciously, probably — to evaluate credibility. Does this user have a “real” profile photo, an anime character, an egg? What is his ratio of followers-to-following? Does his bio suggest expertise in the subject area? In seconds — as few as three, according to one study — you’ve made a judgment.

(For the record, Leonard Nimoy is alive and well! @Mike_FTW is a serial faker.)

Such is the premise of a recent study from Microsoft Research: “As users increasingly access tweets through search,” the paper notes, “they have less information on which to base credibility judgments as compared to consuming content from direct social network connections.”

We trust people in our cultivated social networks. How do we determine the truthiness of a tweet from a stranger? It turns out we evaluate all kinds of clues about credibility, but we’re ultimately pretty bad at separating truth from fiction.

Lead researcher Meredith Ringel Morris surveyed avid Twitter users to identify 32 features of a tweet that help determine credibility. What features were associated with low credibility? The use of non-standard grammar and punctuation; not replacing the default egg account image; using a cartoon or avatar as an account image; and following a large number of users but being followed by few.

Features associated with high credibility “generally concerned the author of the tweet”:

  • Author influence (as measured by follower, retweet, and mention counts)
  • Topical expertise (as established through a Twitter homepage bio)
  • History of on-topic tweeting, pages outside of Twitter, or having a location relevant to the topic of the tweet
  • Reputation (whether an author is someone a user follows, has heard of, or who has an official Twitter account verification seal)

Morris used the data to design two controlled experiments last year with 552 people. Her team essentially recreated the Twitter website and loaded it with made-up tweets that were either true or false but plausible. (The researchers pilot-tested the tweets on fellow staffers to ensure they couldn’t easily detect if a statement was true or false.) She made up gender-neutral names and paired them with either a real photo, a cartoon character, a logo, or egg icon. Respondents were asked to rate the credibility of the tweet, the credibility of the author, and the veracity of the claim.

A false tweet with credible features, or a false tweet from a credible person, might as well be true.

Users attributed a lot of credibility to the user name of a tweet author and, to a lesser extent, the profile image. “Authors with topical names were considered more credible than those with traditional user names, who were in turn considered more credible than those with internet name styles,” the paper says.

So: @AllPolitics > @Alex_Brown > @tenacious27. Users with non-photographic images (a logo, the egg) were viewed as much less credible than those with photos.

If a person’s avatar is a cartoon character, Morris said, it’s easy to dismiss the image as not real. (Sorry, Felix Salmon.) But it’s more difficult to be skeptical of a photo of a real human. “People are actually ascribing a lot of importance to these very small details that are actually very easy to manufacture,” Morris said.

Respondents rated tweets about science significantly more credible than tweets about politics and entertainment, perhaps because science is seen as a more serious and less politicized topic.

What’s more, the most seasoned Twitter users turned out to be the least skeptical. “People who had been users of Twitter longer, people who had more Twitter followers…people who had more expertise with Twitter overall gave higher credibility ratings than those who didn’t,” Morris said. They rated more tweets as true than did Twitter newbies.

It seems counterintuitive, Morris said, but previous research has suggested people put more trust into technology the more they’re familiar with it. (There is evidence to the contrary, too.)

The strongest indicator of a tweet’s credibility? In the survey, respondents gave very high credibility ratings to tweets or retweets that come from a user they trusted. A retweet from a trusted user scored even higher than a tweet from a “verified” user.

Curators have great power and great responsibility.

That affords great power, and great responsibility, to curators. A tweet from a nameless, avatar-less, or otherwise unknown Twitter user is likely to make no impact, but a retweet from, say, Anthony DeRosa or Andy Carvin, gives it great weight.

That’s why seemingly credible users can spread truthy claims. A false tweet with credible features may be just as potent as a true tweet.

Consider the case of Amina Arraf, the “gay girl in Damascus” who turned out to be Tom MacMaster, a straight dude in Edinburgh. By constructing a fake but plausible identity — with a real-sounding name and a real person’s photo — MacMaster was able spin a narrative that fooled even seasoned journalists. MacMaster was able to override our built-in skepticism by constructing the features of credibility we value most.

One of the problems with this study, Morris acknowledges, is that Twitter has been redesigned either once or twice (three times? I’ve lost track) since the study was conducted in 2011. Still, it’s an interesting user-interface question: Can we design websites and search engines to better highlight the details that help us make credibility judgments?

Morris is now conducting a follow-up study to see if she can identify differences across cultures. She is studying Twitter users in the United States and users of the Twitter clone Weibo in China.

POSTED     March 16, 2012, 12:46 p.m.
SHARE THIS STORY
   
Show comments  
Show tags
 
Join the 15,000 who get the freshest future-of-journalism news in our daily email.
Opening up the archives: JSTOR wants to tie a library to the news
Its new site JSTOR Daily highlights interesting research and offers background and context on current events.
Six fresh ideas for news design from a #SNDMakes designathon
New media and legacy media came together at the second weekend-long “hackathon” hosted by the Society for News Design.
Where you get your news depends on where you stand on the issues
A new study by the Pew Research Center examines how Americans’ news consumption habits correlate with where they fall on the political spectrum.
What to read next
1020
tweets
The newsonomics of the millennial moment
The new wave of news startups is aiming at a younger audience. But do legacy media companies have a chance at earning their attention?
803A mixed bag on apps: What The New York Times learned with NYT Opinion and NYT Now
The two apps were part of the paper’s plan to increase digital subscribers through smaller, targeted offerings. Now, with staff cutbacks on the way, one app is being shuttered and the other is being adjusted.
537Watching what happens: The New York Times is making a front-page bet on real-time aggregation
A new homepage feature called “Watching” offers readers a feed of headlines, tweets, and multimedia from around the web.
These stories are our most popular on Twitter over the past 30 days.
See all our most recent pieces ➚
Fuego is our heat-seeking Twitter bot, tracking the links the future-of-journalism crowd is talking about most on Twitter.
Here are a few of the top links Fuego’s currently watching.   Get the full Fuego ➚
Encyclo is our encyclopedia of the future of news, chronicling the key players in journalism’s evolution.
Here are a few of the entries you’ll find in Encyclo.   Get the full Encyclo ➚
Investigative Reporting Workshop
The Blaze
New Jersey Newsroom
Mashable
NewsTilt
Arizona Guardian
ABC News
News Corp
Seattle PostGlobe
FactCheck.org
Next Door Media
Spot.Us