Nieman Foundation at Harvard
HOME
          
LATEST STORY
A Swiss publisher is trying to attract a paying audience with an app sampling stories across publications
ABOUT                    SUBSCRIBE
Oct. 14, 2009, 10 a.m.

How The Huffington Post uses real-time testing to write better headlines

From direct mail to web design, A/B testing is considered a gold standard of user research: Show one version to half your audience and another version to the other half; compare results, and adjust accordingly. Some very cool examples include Google’s obsessive testing of subtle design tweaks and Dustin Curtis’ experiment with direct commands and clickthrough rates. (“You should follow me on Twitter” produced dramatically better results than the less moralizing, “Follow me on Twitter.”)

So here’s something devilishly brilliant: The Huffington Post applies A/B testing to some of its headlines. Readers are randomly shown one of two headlines for the same story. After five minutes, which is enough time for such a high-traffic site, the version with the most clicks becomes the wood that everyone sees.

Headlines have always played the most promotional role in news, charged with selling readers on the articles they adorn, so it only makes sense to apply the best tools of market research to their crafting. Think of it as a more rigorous version of magazines adjusting their covers based on newsstand sales.

Paul Berry, chief technology officer at The Huffington Post, spoke briefly about their real-time headline testing on a panel at the Online News Association conference in San Francisco earlier this month. When I talked to him afterwards, Berry said the system was created inhouse, but he wouldn’t disclose much else about how or how often it’s done. He did say Huffington Post editors have found that placing the author’s name above a headline almost always leads to more clicks than omitting it.

Though it’s unrelated to this A/B testing, The Huffington Post’s new social media editor, Josh Young, has also been soliciting better headlines from readers on Twitter. That’s not as awesomely scientific, but it’s a pretty good use of the crowd.

Studying and responding to users was the theme of Berry’s panel, which also included Steve Dorsey, who conducts qualitative research on reader behavior for the Detroit Free Press, and Eric Brown, homepage planning editor for Yahoo.

Brown said Yahoo closely tracks clickthroughs to measure which content does well with which audiences, as illustrated in his slide below. Yahoo may eventually use that data to serve, say, teenage girls a different version of the front page than they serve to middle-aged men. (Brown’s entire presentation from ONA is here.)

On the topic of audience segmentation, Berry told me that The Huffington Post is considering separate East Coast and West Coast editions. He used the example of morning-after Oscars coverage, when readers flock to slideshows and blog posts about the event. But the story is old news to East Coast readers by noon, when the West Coast is only first logging on, so it would make sense to serve New Yorkers fresher headlines while Californians get their Oscars fix. Identifying readers’ location from their IP address is easy, but coordinating different editions of the site would be a challenge.

POSTED     Oct. 14, 2009, 10 a.m.
SHARE THIS STORY
   
Show comments  
Show tags
 
Join the 15,000 who get the freshest future-of-journalism news in our daily email.
A Swiss publisher is trying to attract a paying audience with an app sampling stories across publications
Tamedia’s 12-App collects the 12 best stories each day from the company’s 20-plus publications.
What does it take to be a “full-service” digital journalism organization? Ask Discourse Media
“We’ve gone down lots of experimental rabbit holes.”
Spain’s Eldiario.es has 18,000 paying members, and its eye on the next several million
“We have a potential of six million readers. You may not convince all six million people to be your socios, but if you learn more about their interests, you can get closer.”
What to read next
0
tweets
Hoping to redefine “trade publication,” Digiday launches Glossy, a vertical to cover disruption in fashion
“I hate the term ‘trade publication,’ because it implies being a boring cheerleader for the industry.”
0Chasing subscriptions over scale, The Athletic wants to turn local sports fandom into a sustainable business — starting in Chicago
“It’s very easy today to be click-driven and produce articles that don’t have a lot of substance or depth and don’t cost that much to produce, but that dynamic is disappointing for fans who want higher-quality content.”
0A year in at Vox, Recode looks at its future: Video, distributed content, more podcasts, and no /
“There’s a huge opportunity to be a widely read, digitally native business site that uses tech as our lens, and I don’t think that’s out there.”
These stories are our most popular on Twitter over the past 30 days.
See all our most recent pieces ➚
Fuego is our heat-seeking Twitter bot, tracking the links the future-of-journalism crowd is talking about most on Twitter.
Here are a few of the top links Fuego’s currently watching.   Get the full Fuego ➚
Encyclo is our encyclopedia of the future of news, chronicling the key players in journalism’s evolution.
Here are a few of the entries you’ll find in Encyclo.   Get the full Encyclo ➚
EveryBlock
WikiLeaks
Placeblogger
Gotham Gazette
Houston Chronicle
MediaNews Group
Mozilla
Spot.Us
U.S. News & World Report
MinnPost
ESPN
The Philadelphia Inquirer & Daily News