Nieman Foundation at Harvard
“Why not be all the way in?” How publishers are using Facebook Instant Articles
ABOUT                    SUBSCRIBE
Oct. 14, 2009, 10 a.m.

How The Huffington Post uses real-time testing to write better headlines

From direct mail to web design, A/B testing is considered a gold standard of user research: Show one version to half your audience and another version to the other half; compare results, and adjust accordingly. Some very cool examples include Google’s obsessive testing of subtle design tweaks and Dustin Curtis’ experiment with direct commands and clickthrough rates. (“You should follow me on Twitter” produced dramatically better results than the less moralizing, “Follow me on Twitter.”)

So here’s something devilishly brilliant: The Huffington Post applies A/B testing to some of its headlines. Readers are randomly shown one of two headlines for the same story. After five minutes, which is enough time for such a high-traffic site, the version with the most clicks becomes the wood that everyone sees.

Headlines have always played the most promotional role in news, charged with selling readers on the articles they adorn, so it only makes sense to apply the best tools of market research to their crafting. Think of it as a more rigorous version of magazines adjusting their covers based on newsstand sales.

Paul Berry, chief technology officer at The Huffington Post, spoke briefly about their real-time headline testing on a panel at the Online News Association conference in San Francisco earlier this month. When I talked to him afterwards, Berry said the system was created inhouse, but he wouldn’t disclose much else about how or how often it’s done. He did say Huffington Post editors have found that placing the author’s name above a headline almost always leads to more clicks than omitting it.

Though it’s unrelated to this A/B testing, The Huffington Post’s new social media editor, Josh Young, has also been soliciting better headlines from readers on Twitter. That’s not as awesomely scientific, but it’s a pretty good use of the crowd.

Studying and responding to users was the theme of Berry’s panel, which also included Steve Dorsey, who conducts qualitative research on reader behavior for the Detroit Free Press, and Eric Brown, homepage planning editor for Yahoo.

Brown said Yahoo closely tracks clickthroughs to measure which content does well with which audiences, as illustrated in his slide below. Yahoo may eventually use that data to serve, say, teenage girls a different version of the front page than they serve to middle-aged men. (Brown’s entire presentation from ONA is here.)

On the topic of audience segmentation, Berry told me that The Huffington Post is considering separate East Coast and West Coast editions. He used the example of morning-after Oscars coverage, when readers flock to slideshows and blog posts about the event. But the story is old news to East Coast readers by noon, when the West Coast is only first logging on, so it would make sense to serve New Yorkers fresher headlines while Californians get their Oscars fix. Identifying readers’ location from their IP address is easy, but coordinating different editions of the site would be a challenge.

POSTED     Oct. 14, 2009, 10 a.m.
Show comments  
Show tags
Join the 15,000 who get the freshest future-of-journalism news in our daily email.
“Why not be all the way in?” How publishers are using Facebook Instant Articles
“If we end up making more money as a publisher, that’s fantastic. I don’t think that’s going to be an afterthought or byproduct; I think there is a way to win from the business perspective.”
How did the GE-branded podcast The Message hit No. 1 on iTunes? In part, by sounding nothing like an ad
“I don’t consider it advertising. It’s a podcast show that just happens to be produced by a brand instead of a network.”
America’s Test Kitchen, “the Consumer Reports of cooking,” wants to grow to new platforms
“We’d like to move to other platforms, particularly as we see the changes in how people consume television.”
What to read next
How one blog helped spark The New York Times’ digital evolution
“I certainly had editors tell me that I shouldn’t be wasting my time on Bird Week. But that was the best part of City Room…We were like unsupervised children.”
572News outlets left and right (and up, down, and center) are embracing virtual reality technology
Among those experimenting is The Wall Street Journal, which plans to open source its 360-degree mobile video and VR technology and hopes to turn VR into more of a mainstay of its storytelling.
502Podcasting in 2015 feels a lot like blogging circa 2004: exciting, evolving, and trouble for incumbents
The same trends we saw a decade ago — professionalization on one hand, platformization on the other — sure seem to be playing out again.
These stories are our most popular on Twitter over the past 30 days.
See all our most recent pieces ➚
Fuego is our heat-seeking Twitter bot, tracking the links the future-of-journalism crowd is talking about most on Twitter.
Here are a few of the top links Fuego’s currently watching.   Get the full Fuego ➚
Encyclo is our encyclopedia of the future of news, chronicling the key players in journalism’s evolution.
Here are a few of the entries you’ll find in Encyclo.   Get the full Encyclo ➚
Center for Public Integrity
The Batavian
Seattle PostGlobe
Los Angeles Times
Global Voices
Zonie Report
The Daily Show
Chicago News Cooperative
New Haven Independent