HOME
          
LATEST STORY
Esquire has a cold: How the magazine is mining its archives with the launch of Esquire Classics
ABOUT                    SUBSCRIBE
Oct. 26, 2010, 10 a.m.

Getting beyond just pageviews: Philly.com’s seven-part equation for measuring online engagement

As web analytics reports become a mainstay of news meetings, there’s a lot of nervousness about how attention to clicks will affect news coverage, and about the perceived incentives to produce high-trafficking junk news. Earlier this week, a web research company released a good-news study arguing that stories about substantive issues like unemployment and mortgage rates can actually bring in more revenue per pageview than celebrity crotch shots.

But two months ago, philly.com, home of the Philadelphia Inquirer and Daily News, began analyzing their web traffic with an “engagement index” — an equation that goes beyond pageviews and into the factors that differentiate a loyal, dedicated reader from a fly-by. It sums up seven different ways that users can show “engagement” with the site, and it looks like this: Σ(Ci + Di + Ri + Li + Bi + Ii + Pi)

I spoke with Chris Meares, senior data analyst at Philly.com, about how the equation works, and what it’s revealed so far about the newspaper’s users on the web. The first step in measuring engagement, Meares explained, is identifying which web behaviors show that users are “engaged.”

Working off a white paper called “Measuring the Unmeasurable: Visitor Engagement” by Eric T. Peterson and Joseph Carrabis, Meares sat down with Ryan Davis, the president of Philly.com, and Wendy Warren, the vice president for content, to hash out the meaning of engagement.

One possibility they considered was measuring engagement simply through how many visitors left comments or shared philly.com content on a social media platform. But that method “would lose a lot of people,” Meares said. “A lot of our users don’t comment or share stories, but we have people — 45 percent — [who] come back more than once a day, and those people are very engaged.”

They ultimately decided on seven categories, each with a particular cutoff:

Ci — Click Index: visits must have at least 6 pageviews, not counting photo galleries

Di — Duration Index: visits must have spend a minimum of 5 minutes on the site

Ri — Recency Index: visits that return daily

Li — Loyalty Index: visits that either are registered at the site or visit it at least three times a week

Bi — Brand Index: visits that come directly to the site by either bookmark or directly typing www.philly.com or come through search engines with keywords like “philly.com” or “inquirer”

Ii — Interaction Index: visits that interact with the site via commenting, forums, etc.

Pi — Participation Index: visits that participate on the site via sharing, uploading pics, stories, videos, etc.

Those are largely the same as in the Peterson/Carrabis white paper, although they use a Feedback Index (“captures qualitative information including propensity to solicit additional information or supply direct feedback”) in place of Philly.com’s Participation Index.

The next step was to track the percentage of overall visits that satisfy each of these categories. What percent of visits to the site lasted at least five minutes? What percent included a comment or other interaction? For instance, in a recent measure of the Click Index, out of 3.9 million total visits, 698,000 were visits where a user clicked through at least six pages — which comes out to a 17.9 percent engagement rate.

As well as paying attention to the engagement rates for each category, Meares also averages the seven individual percentages to create an overall engagement score — the average percent of visits to the site that broadly qualify as “engaged.”

Because the engagement percentage typically goes down as total site pageviews go up (new visitors are, by definition, not loyal ones), Meares multiples the overall engagement percentage by the total number of pageviews to get a estimate of the total number of engaged visits.

Last week, the overall engagement was 31 percent, which translates into an estimated 1.221 million engaged visits.

Month to month, the overall engagement score for philly.com has hovered around 35 percent, Meares said. He also tracks the levels of engagement for different areas of the site, including news, sports, and living. While he said the sports score isn’t actually as high as the 73-percent figure a Philly.com exec gave us last week, it is higher than the average engagement level across the site. In September, sports page visits were 46.6 percent engaged, while news page visits were only 34.4 percent engaged.

Tracking site visits with this level of specificity is time-consuming — Meares says he devotes about a third of his full-time job to analysis of the engagement equation — but it has produced some interesting information. For instance: “We’re definitely seeing the impact of social media and how it provides engaged visitors.” While Google and Yahoo provide a lot of traffic, the visits that they send to Philly.com don’t tend to be engaged. Only 20.34 percent of visits that come through Google are engaged visits. In comparison, 33.64 percent of visits that come via Facebook are engaged.

More broadly, Meares said, tracking engagement allows Philly.com to put traffic data in perspective. If overall traffic for the site is down, but the number of engaged users are up, that still means the site is doing well, Meares said.

Newspapers in other markets have come to Philly.com for advice about measuring engagement on news sites. “It’s a really big challenge,” said Sonia Meisenheimer, digital marketing strategist for the St. Petersburg Times’ tampabay.com. “There’s not an expressed moment of loyalty. You have to basically be a diviner with a divining rod and go around and tap on all these different things.”

Meisenheimer said she found philly.com’s engagement equation “fascinating” but impractical for her requirements. “They can do this because they have a web analytics person full time looking at ‘how do we measure success?'” she said. “We could never sustain the reporting and tracking.”

While the equation might provide interesting feedback to editors, Meisenheimer said she thought the results it produced were too complicated. In a competitive market, businesses want to compare different web outlets on apples-to-apples factors like total audience and local audiences. She said tampabay.com would measure engagement in a much simpler way, focusing on registration numbers and their brand index, or how many people come to the site through a bookmark or by searching for terms like “St. Pete Times” or “tampabay.com.”

Internally, she said the most important number for tampabay.com is simply revenue per unique visitor. “My question to philly.com is: how does this help you make more money? Because I don’t see that in the equation.”

POSTED     Oct. 26, 2010, 10 a.m.
SHARE THIS STORY
   
Show comments  
Show tags
 
Join the 15,000 who get the freshest future-of-journalism news in our daily email.
Esquire has a cold: How the magazine is mining its archives with the launch of Esquire Classics
“We’re continuing our experiments with seeing what kinds of great archival stories people want to read and what formats seem to be most popular.”
The Atlantic redesigns, trading clutter and density for refinement
It wants to be a “real-time magazine” on the web, connected to its print heritage. But stripping out the visual noise won’t please everyone.
Getting beyond “public radio voice”: Finding and decoding identity on the air
Public radio voice or public radio voices? Figuring out how different identities fit together on the airwaves is a challenge for many journalists.
What to read next
2439
tweets
The Economist’s Tom Standage on digital strategy and the limits of a model based on advertising
“The Economist has taken the view that advertising is nice, and we’ll certainly take money where we can get it, but we’re pretty much expecting it to go away.”
579What USA Today Sports learned covering the Final Four on Periscope and Snapchat
These new platforms are optimized for realtime news on phones, but there are lots of questions for news organizations — from what content to share to how to measure their effectiveness.
410Journalists shouldn’t lose their rights in their move to private platforms
The shift to distributed content means concepts like fair use are increasingly in the hands of private companies — like SoundCloud.
These stories are our most popular on Twitter over the past 30 days.
See all our most recent pieces ➚
Encyclo is our encyclopedia of the future of news, chronicling the key players in journalism’s evolution.
Here are a few of the entries you’ll find in Encyclo.   Get the full Encyclo ➚
Amazon
The UpTake
Placeblogger
Honolulu Civil Beat
The Daily Voice
FiveThirtyEight
Financial Times
Demand Media
American Public Media
The Times of London
NBC News
MediaNews Group