Nieman Foundation at Harvard
Medium partners with publications like The Awl and Fusion, and more native ads are on the way
ABOUT                    SUBSCRIBE
Oct. 26, 2010, 10 a.m.

Getting beyond just pageviews:’s seven-part equation for measuring online engagement

As web analytics reports become a mainstay of news meetings, there’s a lot of nervousness about how attention to clicks will affect news coverage, and about the perceived incentives to produce high-trafficking junk news. Earlier this week, a web research company released a good-news study arguing that stories about substantive issues like unemployment and mortgage rates can actually bring in more revenue per pageview than celebrity crotch shots.

But two months ago,, home of the Philadelphia Inquirer and Daily News, began analyzing their web traffic with an “engagement index” — an equation that goes beyond pageviews and into the factors that differentiate a loyal, dedicated reader from a fly-by. It sums up seven different ways that users can show “engagement” with the site, and it looks like this: Σ(Ci + Di + Ri + Li + Bi + Ii + Pi)

I spoke with Chris Meares, senior data analyst at, about how the equation works, and what it’s revealed so far about the newspaper’s users on the web. The first step in measuring engagement, Meares explained, is identifying which web behaviors show that users are “engaged.”

Working off a white paper called “Measuring the Unmeasurable: Visitor Engagement” by Eric T. Peterson and Joseph Carrabis, Meares sat down with Ryan Davis, the president of, and Wendy Warren, the vice president for content, to hash out the meaning of engagement.

One possibility they considered was measuring engagement simply through how many visitors left comments or shared content on a social media platform. But that method “would lose a lot of people,” Meares said. “A lot of our users don’t comment or share stories, but we have people — 45 percent — [who] come back more than once a day, and those people are very engaged.”

They ultimately decided on seven categories, each with a particular cutoff:

Ci — Click Index: visits must have at least 6 pageviews, not counting photo galleries

Di — Duration Index: visits must have spend a minimum of 5 minutes on the site

Ri — Recency Index: visits that return daily

Li — Loyalty Index: visits that either are registered at the site or visit it at least three times a week

Bi — Brand Index: visits that come directly to the site by either bookmark or directly typing or come through search engines with keywords like “” or “inquirer”

Ii — Interaction Index: visits that interact with the site via commenting, forums, etc.

Pi — Participation Index: visits that participate on the site via sharing, uploading pics, stories, videos, etc.

Those are largely the same as in the Peterson/Carrabis white paper, although they use a Feedback Index (“captures qualitative information including propensity to solicit additional information or supply direct feedback”) in place of’s Participation Index.

The next step was to track the percentage of overall visits that satisfy each of these categories. What percent of visits to the site lasted at least five minutes? What percent included a comment or other interaction? For instance, in a recent measure of the Click Index, out of 3.9 million total visits, 698,000 were visits where a user clicked through at least six pages — which comes out to a 17.9 percent engagement rate.

As well as paying attention to the engagement rates for each category, Meares also averages the seven individual percentages to create an overall engagement score — the average percent of visits to the site that broadly qualify as “engaged.”

Because the engagement percentage typically goes down as total site pageviews go up (new visitors are, by definition, not loyal ones), Meares multiples the overall engagement percentage by the total number of pageviews to get a estimate of the total number of engaged visits.

Last week, the overall engagement was 31 percent, which translates into an estimated 1.221 million engaged visits.

Month to month, the overall engagement score for has hovered around 35 percent, Meares said. He also tracks the levels of engagement for different areas of the site, including news, sports, and living. While he said the sports score isn’t actually as high as the 73-percent figure a exec gave us last week, it is higher than the average engagement level across the site. In September, sports page visits were 46.6 percent engaged, while news page visits were only 34.4 percent engaged.

Tracking site visits with this level of specificity is time-consuming — Meares says he devotes about a third of his full-time job to analysis of the engagement equation — but it has produced some interesting information. For instance: “We’re definitely seeing the impact of social media and how it provides engaged visitors.” While Google and Yahoo provide a lot of traffic, the visits that they send to don’t tend to be engaged. Only 20.34 percent of visits that come through Google are engaged visits. In comparison, 33.64 percent of visits that come via Facebook are engaged.

More broadly, Meares said, tracking engagement allows to put traffic data in perspective. If overall traffic for the site is down, but the number of engaged users are up, that still means the site is doing well, Meares said.

Newspapers in other markets have come to for advice about measuring engagement on news sites. “It’s a really big challenge,” said Sonia Meisenheimer, digital marketing strategist for the St. Petersburg Times’ “There’s not an expressed moment of loyalty. You have to basically be a diviner with a divining rod and go around and tap on all these different things.”

Meisenheimer said she found’s engagement equation “fascinating” but impractical for her requirements. “They can do this because they have a web analytics person full time looking at ‘how do we measure success?'” she said. “We could never sustain the reporting and tracking.”

While the equation might provide interesting feedback to editors, Meisenheimer said she thought the results it produced were too complicated. In a competitive market, businesses want to compare different web outlets on apples-to-apples factors like total audience and local audiences. She said would measure engagement in a much simpler way, focusing on registration numbers and their brand index, or how many people come to the site through a bookmark or by searching for terms like “St. Pete Times” or “”

Internally, she said the most important number for is simply revenue per unique visitor. “My question to is: how does this help you make more money? Because I don’t see that in the equation.”

POSTED     Oct. 26, 2010, 10 a.m.
Show comments  
Show tags
Join the 15,000 who get the freshest future-of-journalism news in our daily email.
Medium partners with publications like The Awl and Fusion, and more native ads are on the way
Medium is rolling out a slew of changes, including publisher partnerships, updated apps, and plans for advertising.
4 takeaways from The New York Times’ new digital strategy memo
With a renewed focus on subscriptions, the Times believes it can double its digital revenue to $800 million in 2020.
Get AMP’d: Here’s what publishers need to know about Google’s new plan to speed up your website
The speed gains are very real. But do publishers want to trade in the open space of what we’ve known as the web for yet another platform they have little control over?
What to read next
What happened after 7 news sites got rid of reader comments
Recode, Reuters, Popular Science, The Week, Mic, The Verge, and USA Today’s FTW have all shut off reader comments in the past year. Here’s how they’re all using social media to encourage reader discussion.
699Facebook woos journalists with Signal, a dashboard to gather news across Facebook and Instagram
Signal helps journalists find, source, and embed content from Facebook and Instagram.
587Get AMP’d: Here’s what publishers need to know about Google’s new plan to speed up your website
The speed gains are very real. But do publishers want to trade in the open space of what we’ve known as the web for yet another platform they have little control over?
These stories are our most popular on Twitter over the past 30 days.
See all our most recent pieces ➚
Fuego is our heat-seeking Twitter bot, tracking the links the future-of-journalism crowd is talking about most on Twitter.
Here are a few of the top links Fuego’s currently watching.   Get the full Fuego ➚
Encyclo is our encyclopedia of the future of news, chronicling the key players in journalism’s evolution.
Here are a few of the entries you’ll find in Encyclo.   Get the full Encyclo ➚
Talking Points Memo
Daily Kos
The Batavian
The Daily Beast
The Tyee
Chicago Tribune