HOME
          
LATEST STORY
Ken Doctor: The New York Times’ financials show the transition to digital accelerating
ABOUT                    SUBSCRIBE
Oct. 26, 2010, 10 a.m.

Getting beyond just pageviews: Philly.com’s seven-part equation for measuring online engagement

As web analytics reports become a mainstay of news meetings, there’s a lot of nervousness about how attention to clicks will affect news coverage, and about the perceived incentives to produce high-trafficking junk news. Earlier this week, a web research company released a good-news study arguing that stories about substantive issues like unemployment and mortgage rates can actually bring in more revenue per pageview than celebrity crotch shots.

But two months ago, philly.com, home of the Philadelphia Inquirer and Daily News, began analyzing their web traffic with an “engagement index” — an equation that goes beyond pageviews and into the factors that differentiate a loyal, dedicated reader from a fly-by. It sums up seven different ways that users can show “engagement” with the site, and it looks like this: Σ(Ci + Di + Ri + Li + Bi + Ii + Pi)

I spoke with Chris Meares, senior data analyst at Philly.com, about how the equation works, and what it’s revealed so far about the newspaper’s users on the web. The first step in measuring engagement, Meares explained, is identifying which web behaviors show that users are “engaged.”

Working off a white paper called “Measuring the Unmeasurable: Visitor Engagement” by Eric T. Peterson and Joseph Carrabis, Meares sat down with Ryan Davis, the president of Philly.com, and Wendy Warren, the vice president for content, to hash out the meaning of engagement.

One possibility they considered was measuring engagement simply through how many visitors left comments or shared philly.com content on a social media platform. But that method “would lose a lot of people,” Meares said. “A lot of our users don’t comment or share stories, but we have people — 45 percent — [who] come back more than once a day, and those people are very engaged.”

They ultimately decided on seven categories, each with a particular cutoff:

Ci — Click Index: visits must have at least 6 pageviews, not counting photo galleries

Di — Duration Index: visits must have spend a minimum of 5 minutes on the site

Ri — Recency Index: visits that return daily

Li — Loyalty Index: visits that either are registered at the site or visit it at least three times a week

Bi — Brand Index: visits that come directly to the site by either bookmark or directly typing www.philly.com or come through search engines with keywords like “philly.com” or “inquirer”

Ii — Interaction Index: visits that interact with the site via commenting, forums, etc.

Pi — Participation Index: visits that participate on the site via sharing, uploading pics, stories, videos, etc.

Those are largely the same as in the Peterson/Carrabis white paper, although they use a Feedback Index (“captures qualitative information including propensity to solicit additional information or supply direct feedback”) in place of Philly.com’s Participation Index.

The next step was to track the percentage of overall visits that satisfy each of these categories. What percent of visits to the site lasted at least five minutes? What percent included a comment or other interaction? For instance, in a recent measure of the Click Index, out of 3.9 million total visits, 698,000 were visits where a user clicked through at least six pages — which comes out to a 17.9 percent engagement rate.

As well as paying attention to the engagement rates for each category, Meares also averages the seven individual percentages to create an overall engagement score — the average percent of visits to the site that broadly qualify as “engaged.”

Because the engagement percentage typically goes down as total site pageviews go up (new visitors are, by definition, not loyal ones), Meares multiples the overall engagement percentage by the total number of pageviews to get a estimate of the total number of engaged visits.

Last week, the overall engagement was 31 percent, which translates into an estimated 1.221 million engaged visits.

Month to month, the overall engagement score for philly.com has hovered around 35 percent, Meares said. He also tracks the levels of engagement for different areas of the site, including news, sports, and living. While he said the sports score isn’t actually as high as the 73-percent figure a Philly.com exec gave us last week, it is higher than the average engagement level across the site. In September, sports page visits were 46.6 percent engaged, while news page visits were only 34.4 percent engaged.

Tracking site visits with this level of specificity is time-consuming — Meares says he devotes about a third of his full-time job to analysis of the engagement equation — but it has produced some interesting information. For instance: “We’re definitely seeing the impact of social media and how it provides engaged visitors.” While Google and Yahoo provide a lot of traffic, the visits that they send to Philly.com don’t tend to be engaged. Only 20.34 percent of visits that come through Google are engaged visits. In comparison, 33.64 percent of visits that come via Facebook are engaged.

More broadly, Meares said, tracking engagement allows Philly.com to put traffic data in perspective. If overall traffic for the site is down, but the number of engaged users are up, that still means the site is doing well, Meares said.

Newspapers in other markets have come to Philly.com for advice about measuring engagement on news sites. “It’s a really big challenge,” said Sonia Meisenheimer, digital marketing strategist for the St. Petersburg Times’ tampabay.com. “There’s not an expressed moment of loyalty. You have to basically be a diviner with a divining rod and go around and tap on all these different things.”

Meisenheimer said she found philly.com’s engagement equation “fascinating” but impractical for her requirements. “They can do this because they have a web analytics person full time looking at ‘how do we measure success?'” she said. “We could never sustain the reporting and tracking.”

While the equation might provide interesting feedback to editors, Meisenheimer said she thought the results it produced were too complicated. In a competitive market, businesses want to compare different web outlets on apples-to-apples factors like total audience and local audiences. She said tampabay.com would measure engagement in a much simpler way, focusing on registration numbers and their brand index, or how many people come to the site through a bookmark or by searching for terms like “St. Pete Times” or “tampabay.com.”

Internally, she said the most important number for tampabay.com is simply revenue per unique visitor. “My question to philly.com is: how does this help you make more money? Because I don’t see that in the equation.”

POSTED     Oct. 26, 2010, 10 a.m.
SHARE THIS STORY
   
Show comments  
Show tags
 
Join the 15,000 who get the freshest future-of-journalism news in our daily email.
Ken Doctor: The New York Times’ financials show the transition to digital accelerating
The numbers may look flat, but they contain a continuing set of ups and downs. Up next: executing on a year’s worth of launches.
Before the “teaching hospital model” of journalism education: 5 questions to ask
It’ll take a new generation of academic leadership — willing to incur the wrath of faculty, the greater university, alumni, industry, and analysts — to break through the old ways we train journalists.
Controlled chaos: As journalism and documentary film converge in digital, what lessons can they share?
Old and new media types from journalism, documentary, and technology backgrounds gathered at MIT to share practices and discuss mutual concerns.
What to read next
1020
tweets
The newsonomics of the millennial moment
The new wave of news startups is aiming at a younger audience. But do legacy media companies have a chance at earning their attention?
803A mixed bag on apps: What The New York Times learned with NYT Opinion and NYT Now
The two apps were part of the paper’s plan to increase digital subscribers through smaller, targeted offerings. Now, with staff cutbacks on the way, one app is being shuttered and the other is being adjusted.
413The new Vox daily email, explained
The company’s newsletter, Vox Sentences, enters an increasingly crowded inbox. Can concise writing and smart aggregation on the day’s news help expand their audience?
These stories are our most popular on Twitter over the past 30 days.
See all our most recent pieces ➚
Encyclo is our encyclopedia of the future of news, chronicling the key players in journalism’s evolution.
Here are a few of the entries you’ll find in Encyclo.   Get the full Encyclo ➚
Tampa Bay Times
GlobalPost
Press+
New Jersey Newsroom
The Globe and Mail
The New Republic
Bloomberg
New York
Ars Technica
Craigslist
The Philadelphia Inquirer & Daily News
The Daily Voice