HOME
          
LATEST STORY
Before the “teaching model” of journalism education: 5 questions to ask
ABOUT                    SUBSCRIBE
Feb. 6, 2014, 10:33 a.m.
attmingoals[1][1]

You won’t believe Upworthy’s new way of measuring audience engagement until you read it

“Attention minutes” combine traditional metrics with measurements aimed at figuring out how much time readers really spend with content.

Upworthy wants to be known as the company that shares a massive amount of socially conscious content across the web. What Upworthy is widely known for, however, is perfecting the art of the clickbait, curiosity gap headline. (On the front page now: “The Perfect Reply A Girl Can Give To The Question ‘What’s Your Favorite Position?'” “What If Everyone Who Reacted Negatively To A Super Bowl Ad Knew The Facts? They’d Learn This.” “A 16-Year-Old Explains Why Everything You Thought You Knew About Beauty May Be Wrong. With Math.”)

The company denies that they’re the main culprit, and some initial research by The Atlantic’s Robinson Meyer suggests that may be true. But now, they’re taking steps toward measuring their performance through something deeper than clicks.

This morning, Upworthy announced it would be using a new metric — “attention minutes” — as its primary tool to determine how they’re doing. The goal is to blend traditional eyeball-counting metrics with figures that more accurately measure engagement or how much the audience actually likes the content that they’re making. And they’ll be releasing their source code so other publishers can use it.

We love thinking this way because it rewards us for sharing content that people really enjoy and find valuable — not just stuff they click on a lot. It may mean that we don’t do quite as well on uniques or pageviews, but that’s a tradeoff we’re happy to make because this is a metric focused on real user satisfaction.

Upworthy says they decided to build attention minutes in the fall out of frustration with available analytics, either because they measure the wrong things or because they measured them poorly. For example, while Google Analytics’ time-on-site metric seems like it should provide something like what Upworthy was looking for, it was often misrepresentative, calculating at one point that the average Upworthy user spent 21 minutes on site. (“We’re good, but we know we’re not that good,” they write.)

Of course, Upworthy isn’t the only publisher feeling disatisfied with the mighty pageview and its many successors. Former Knight-Mozilla OpenNews fellow Brian Abelson, who now works at Enigma, spent part of his fellowship at The New York Times working on “pageviews above replacement,” a method of looking at basic traffic results without the confounding factor of internal promotion. James Robinson, director of news analytics at the Times, had this to say about metrics in his Nieman Lab prediction for news in 2014:

But for all types of newsrooms, developing a more qualitative understanding of audiences will be increasingly important. As publishers continue to invest in longform journalism and experiment with new types of storytelling, questions will naturally arise about the ways in which people actually read and engage with these pieces, beyond the feel-good tally of how many visitors they attract.

As Upworthy acknowledges, other media companies have also begun to make moves towards more meaningful metrics. “There are definitely others looking at how to improve metrics,” writes Upworthy CEO Eli Pariser in an email, citing YouTube’s “Time Watched,” Medium’s “Total Time Reading,” and Chartbeat’s “Average Engaged Time.” (Chartbeat responded to Upworthy’s announcement on their own blog, calling the metric “another big win.”)

“They all have a similar hunch,” Pariser said. “As we’ve been building it out it’s been nice to see additional traction on this front. It gives us a sense we’re not way out in left field.”

Attention minutes are calculated using an assemblage of what Upworthy calls “signals.” Pariser says they started out by asking, “What are the ways you can know if it’s a sad, lonely, ignored tab or if someone is watching intently?”

Some of these signals include the length of time a browser tab has been open, how long a video player has been running, and the movement of the mouse on screen. “This method doesn’t store raw information — it follows a stream of events and every few seconds asks, ‘Is this person still paying attention?'” Pariser said. “If they are, that sends a signal to a data warehouse that says, ‘Chalk up a few more attention seconds for this post.'” Together with other information, including pageviews, Upworthy hopes to get a sense of what content is most delighting and engrossing viewers, not just briefly capturing their attention.

This chart is meant to illustrate how the signals that make up attention minutes present a significantly different portrait of a story than pageviews do:

upworhty1

Upworthy plans to release the source code “in the coming months.” Pariser says Upworthy has already begun to gather valuable information about how people consume content — like that “the relationship between the length of a video and how much people watch isn’t what we though it would be,” he says. They’ll be publishing more of that information in 2014, too.

The overarching hope is that by improving the quality of your metrics, you can improve the quality of your work. Campbell’s Law puts it well, if academically:

The more any quantitative social indicator (or even some qualitative indicator) is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.

If you measure performance in pageviews, you encourage slideshows. If you measure performance by social shares, you encourage clickbait headlines and giant Like buttons. Finding a metric that lines up with a publisher’s goals is one of the most important things it can do to encourage better work, and Upworthy thinks this is a step closer to that alignment for their needs.

“The reason that we’re focused on this metric is not just because we think it’s good for us, but because we think it would serve everybody better if we moved away from some of the metrics that people have been using,” Pariser says. “Pageviews can be bought, but this…is a truer measure of whether you’re delivering something meaningful and valuable.”

POSTED     Feb. 6, 2014, 10:33 a.m.
SHARE THIS STORY
   
Show comments  
Show tags
 
Join the 15,000 who get the freshest future-of-journalism news in our daily email.
Before the “teaching model” of journalism education: 5 questions to ask
It’ll take a new generation of academic leadership — willing to incur the wrath of faculty, the greater university, alumni, industry, and analysts — to break through the old ways we train journalists.
Controlled chaos: As journalism and documentary film converge in digital, what lessons can they share?
Old and new media types from journalism, documentary, and technology backgrounds gathered at MIT to share practices and discuss mutual concerns.
The near future of First Look’s next site, Racket, looks fuzzy
The site, promised as a “satirical approach to American politics and culture,” was set to launch this month, but now it’s unclear when or if it’ll get off the ground.
What to read next
1020
tweets
The newsonomics of the millennial moment
The new wave of news startups is aiming at a younger audience. But do legacy media companies have a chance at earning their attention?
803A mixed bag on apps: What The New York Times learned with NYT Opinion and NYT Now
The two apps were part of the paper’s plan to increase digital subscribers through smaller, targeted offerings. Now, with staff cutbacks on the way, one app is being shuttered and the other is being adjusted.
413The new Vox daily email, explained
The company’s newsletter, Vox Sentences, enters an increasingly crowded inbox. Can concise writing and smart aggregation on the day’s news help expand their audience?
These stories are our most popular on Twitter over the past 30 days.
See all our most recent pieces ➚
Encyclo is our encyclopedia of the future of news, chronicling the key players in journalism’s evolution.
Here are a few of the entries you’ll find in Encyclo.   Get the full Encyclo ➚
American Public Media
New England Center for Investigative Reporting
ReadWrite
Honolulu Civil Beat
SF Appeal
PBS
Wikipedia
Connecticut Mirror
Salon
The Bay Citizen
O Globo
WyoFile