HOME
          
LATEST STORY
The newsonomics of MLB’s pioneering mobile experience
ABOUT                    SUBSCRIBE
Feb. 6, 2014, 10:33 a.m.
attmingoals[1][1]

You won’t believe Upworthy’s new way of measuring audience engagement until you read it

“Attention minutes” combine traditional metrics with measurements aimed at figuring out how much time readers really spend with content.

Upworthy wants to be known as the company that shares a massive amount of socially conscious content across the web. What Upworthy is widely known for, however, is perfecting the art of the clickbait, curiosity gap headline. (On the front page now: “The Perfect Reply A Girl Can Give To The Question ‘What’s Your Favorite Position?'” “What If Everyone Who Reacted Negatively To A Super Bowl Ad Knew The Facts? They’d Learn This.” “A 16-Year-Old Explains Why Everything You Thought You Knew About Beauty May Be Wrong. With Math.”)

The company denies that they’re the main culprit, and some initial research by The Atlantic’s Robinson Meyer suggests that may be true. But now, they’re taking steps toward measuring their performance through something deeper than clicks.

This morning, Upworthy announced it would be using a new metric — “attention minutes” — as its primary tool to determine how they’re doing. The goal is to blend traditional eyeball-counting metrics with figures that more accurately measure engagement or how much the audience actually likes the content that they’re making. And they’ll be releasing their source code so other publishers can use it.

We love thinking this way because it rewards us for sharing content that people really enjoy and find valuable — not just stuff they click on a lot. It may mean that we don’t do quite as well on uniques or pageviews, but that’s a tradeoff we’re happy to make because this is a metric focused on real user satisfaction.

Upworthy says they decided to build attention minutes in the fall out of frustration with available analytics, either because they measure the wrong things or because they measured them poorly. For example, while Google Analytics’ time-on-site metric seems like it should provide something like what Upworthy was looking for, it was often misrepresentative, calculating at one point that the average Upworthy user spent 21 minutes on site. (“We’re good, but we know we’re not that good,” they write.)

Of course, Upworthy isn’t the only publisher feeling disatisfied with the mighty pageview and its many successors. Former Knight-Mozilla OpenNews fellow Brian Abelson, who now works at Enigma, spent part of his fellowship at The New York Times working on “pageviews above replacement,” a method of looking at basic traffic results without the confounding factor of internal promotion. James Robinson, director of news analytics at the Times, had this to say about metrics in his Nieman Lab prediction for news in 2014:

But for all types of newsrooms, developing a more qualitative understanding of audiences will be increasingly important. As publishers continue to invest in longform journalism and experiment with new types of storytelling, questions will naturally arise about the ways in which people actually read and engage with these pieces, beyond the feel-good tally of how many visitors they attract.

As Upworthy acknowledges, other media companies have also begun to make moves towards more meaningful metrics. “There are definitely others looking at how to improve metrics,” writes Upworthy CEO Eli Pariser in an email, citing YouTube’s “Time Watched,” Medium’s “Total Time Reading,” and Chartbeat’s “Average Engaged Time.” (Chartbeat responded to Upworthy’s announcement on their own blog, calling the metric “another big win.”)

“They all have a similar hunch,” Pariser said. “As we’ve been building it out it’s been nice to see additional traction on this front. It gives us a sense we’re not way out in left field.”

Attention minutes are calculated using an assemblage of what Upworthy calls “signals.” Pariser says they started out by asking, “What are the ways you can know if it’s a sad, lonely, ignored tab or if someone is watching intently?”

Some of these signals include the length of time a browser tab has been open, how long a video player has been running, and the movement of the mouse on screen. “This method doesn’t store raw information — it follows a stream of events and every few seconds asks, ‘Is this person still paying attention?'” Pariser said. “If they are, that sends a signal to a data warehouse that says, ‘Chalk up a few more attention seconds for this post.'” Together with other information, including pageviews, Upworthy hopes to get a sense of what content is most delighting and engrossing viewers, not just briefly capturing their attention.

This chart is meant to illustrate how the signals that make up attention minutes present a significantly different portrait of a story than pageviews do:

upworhty1

Upworthy plans to release the source code “in the coming months.” Pariser says Upworthy has already begun to gather valuable information about how people consume content — like that “the relationship between the length of a video and how much people watch isn’t what we though it would be,” he says. They’ll be publishing more of that information in 2014, too.

The overarching hope is that by improving the quality of your metrics, you can improve the quality of your work. Campbell’s Law puts it well, if academically:

The more any quantitative social indicator (or even some qualitative indicator) is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.

If you measure performance in pageviews, you encourage slideshows. If you measure performance by social shares, you encourage clickbait headlines and giant Like buttons. Finding a metric that lines up with a publisher’s goals is one of the most important things it can do to encourage better work, and Upworthy thinks this is a step closer to that alignment for their needs.

“The reason that we’re focused on this metric is not just because we think it’s good for us, but because we think it would serve everybody better if we moved away from some of the metrics that people have been using,” Pariser says. “Pageviews can be bought, but this…is a truer measure of whether you’re delivering something meaningful and valuable.”

POSTED     Feb. 6, 2014, 10:33 a.m.
SHARE THIS STORY
   
Show comments  
Show tags
 
Join the 15,000 who get the freshest future-of-journalism news in our daily email.
The newsonomics of MLB’s pioneering mobile experience
Running a sports league and running a news operation aren’t the same thing. But there are lessons to be learned from baseball’s success in navigating mobile.
Why The New York Times built a tool for crowdsourced time travel
Madison, a new tool that asks readers to help identify ads in the Times archives, is part of a new open source platform for crowdsourcing built by the company’s R&D Lab.
Opening up the archives: JSTOR wants to tie a library to the news
Its new site JSTOR Daily highlights interesting research and offers background and context on current events.
What to read next
1020
tweets
The newsonomics of the millennial moment
The new wave of news startups is aiming at a younger audience. But do legacy media companies have a chance at earning their attention?
803A mixed bag on apps: What The New York Times learned with NYT Opinion and NYT Now
The two apps were part of the paper’s plan to increase digital subscribers through smaller, targeted offerings. Now, with staff cutbacks on the way, one app is being shuttered and the other is being adjusted.
413The new Vox daily email, explained
The company’s newsletter, Vox Sentences, enters an increasingly crowded inbox. Can concise writing and smart aggregation on the day’s news help expand their audience?
These stories are our most popular on Twitter over the past 30 days.
See all our most recent pieces ➚
Encyclo is our encyclopedia of the future of news, chronicling the key players in journalism’s evolution.
Here are a few of the entries you’ll find in Encyclo.   Get the full Encyclo ➚
Outside.in
Craigslist
OpenFile
Grist
PBS
Journal Register Co.
Lens
American Public Media
MediaNews Group
Investigative Reporting Workshop
NPR
McClatchy