HOME
          
LATEST STORY
An ad blocker for tragedies: How news sites handle content around sensitive stories
ABOUT                    SUBSCRIBE
Oct. 26, 2010, 10 a.m.

Getting beyond just pageviews: Philly.com’s seven-part equation for measuring online engagement

As web analytics reports become a mainstay of news meetings, there’s a lot of nervousness about how attention to clicks will affect news coverage, and about the perceived incentives to produce high-trafficking junk news. Earlier this week, a web research company released a good-news study arguing that stories about substantive issues like unemployment and mortgage rates can actually bring in more revenue per pageview than celebrity crotch shots.

But two months ago, philly.com, home of the Philadelphia Inquirer and Daily News, began analyzing their web traffic with an “engagement index” — an equation that goes beyond pageviews and into the factors that differentiate a loyal, dedicated reader from a fly-by. It sums up seven different ways that users can show “engagement” with the site, and it looks like this: Σ(Ci + Di + Ri + Li + Bi + Ii + Pi)

I spoke with Chris Meares, senior data analyst at Philly.com, about how the equation works, and what it’s revealed so far about the newspaper’s users on the web. The first step in measuring engagement, Meares explained, is identifying which web behaviors show that users are “engaged.”

Working off a white paper called “Measuring the Unmeasurable: Visitor Engagement” by Eric T. Peterson and Joseph Carrabis, Meares sat down with Ryan Davis, the president of Philly.com, and Wendy Warren, the vice president for content, to hash out the meaning of engagement.

One possibility they considered was measuring engagement simply through how many visitors left comments or shared philly.com content on a social media platform. But that method “would lose a lot of people,” Meares said. “A lot of our users don’t comment or share stories, but we have people — 45 percent — [who] come back more than once a day, and those people are very engaged.”

They ultimately decided on seven categories, each with a particular cutoff:

Ci — Click Index: visits must have at least 6 pageviews, not counting photo galleries

Di — Duration Index: visits must have spend a minimum of 5 minutes on the site

Ri — Recency Index: visits that return daily

Li — Loyalty Index: visits that either are registered at the site or visit it at least three times a week

Bi — Brand Index: visits that come directly to the site by either bookmark or directly typing www.philly.com or come through search engines with keywords like “philly.com” or “inquirer”

Ii — Interaction Index: visits that interact with the site via commenting, forums, etc.

Pi — Participation Index: visits that participate on the site via sharing, uploading pics, stories, videos, etc.

Those are largely the same as in the Peterson/Carrabis white paper, although they use a Feedback Index (“captures qualitative information including propensity to solicit additional information or supply direct feedback”) in place of Philly.com’s Participation Index.

The next step was to track the percentage of overall visits that satisfy each of these categories. What percent of visits to the site lasted at least five minutes? What percent included a comment or other interaction? For instance, in a recent measure of the Click Index, out of 3.9 million total visits, 698,000 were visits where a user clicked through at least six pages — which comes out to a 17.9 percent engagement rate.

As well as paying attention to the engagement rates for each category, Meares also averages the seven individual percentages to create an overall engagement score — the average percent of visits to the site that broadly qualify as “engaged.”

Because the engagement percentage typically goes down as total site pageviews go up (new visitors are, by definition, not loyal ones), Meares multiples the overall engagement percentage by the total number of pageviews to get a estimate of the total number of engaged visits.

Last week, the overall engagement was 31 percent, which translates into an estimated 1.221 million engaged visits.

Month to month, the overall engagement score for philly.com has hovered around 35 percent, Meares said. He also tracks the levels of engagement for different areas of the site, including news, sports, and living. While he said the sports score isn’t actually as high as the 73-percent figure a Philly.com exec gave us last week, it is higher than the average engagement level across the site. In September, sports page visits were 46.6 percent engaged, while news page visits were only 34.4 percent engaged.

Tracking site visits with this level of specificity is time-consuming — Meares says he devotes about a third of his full-time job to analysis of the engagement equation — but it has produced some interesting information. For instance: “We’re definitely seeing the impact of social media and how it provides engaged visitors.” While Google and Yahoo provide a lot of traffic, the visits that they send to Philly.com don’t tend to be engaged. Only 20.34 percent of visits that come through Google are engaged visits. In comparison, 33.64 percent of visits that come via Facebook are engaged.

More broadly, Meares said, tracking engagement allows Philly.com to put traffic data in perspective. If overall traffic for the site is down, but the number of engaged users are up, that still means the site is doing well, Meares said.

Newspapers in other markets have come to Philly.com for advice about measuring engagement on news sites. “It’s a really big challenge,” said Sonia Meisenheimer, digital marketing strategist for the St. Petersburg Times’ tampabay.com. “There’s not an expressed moment of loyalty. You have to basically be a diviner with a divining rod and go around and tap on all these different things.”

Meisenheimer said she found philly.com’s engagement equation “fascinating” but impractical for her requirements. “They can do this because they have a web analytics person full time looking at ‘how do we measure success?'” she said. “We could never sustain the reporting and tracking.”

While the equation might provide interesting feedback to editors, Meisenheimer said she thought the results it produced were too complicated. In a competitive market, businesses want to compare different web outlets on apples-to-apples factors like total audience and local audiences. She said tampabay.com would measure engagement in a much simpler way, focusing on registration numbers and their brand index, or how many people come to the site through a bookmark or by searching for terms like “St. Pete Times” or “tampabay.com.”

Internally, she said the most important number for tampabay.com is simply revenue per unique visitor. “My question to philly.com is: how does this help you make more money? Because I don’t see that in the equation.”

POSTED     Oct. 26, 2010, 10 a.m.
SHARE THIS STORY
   
Show comments  
Show tags
 
Join the 15,000 who get the freshest future-of-journalism news in our daily email.
An ad blocker for tragedies: How news sites handle content around sensitive stories
For stories like the Germanwings plane crash, The New York Times and many other publishers flip a switch to remove ads to avoid unwanted connections.
Newsonomics: BuzzFeed and The New York Times play Facebook’s ubiquity game
The ubiquity game has different rules for digital startups than for legacy businesses. But for both, figuring out the right relationship with Facebook is key to their audience strategies.
Jeff Israely: Good content marketing benefits from a smart publisher’s touch
Our startup correspondent, building Worldcrunch in Paris, on the thinking behind its operation’s pivot: “The smart brands know they’ll lose your attention if they use this new publishing power simply to push their merchandise.”
What to read next
2481
tweets
Millennials say keeping up with the news is important to them — but good luck getting them to pay for it
The new report from the Media Insight Project looks at millennials’ habits and attitudes toward news consumption: “I really wouldn’t pay for any type of news because as a citizen it’s my right to know the news.”
926The next stage in the battle for our attention: Our wrists
News companies have moved from print dollars to digital dimes to mobile pennies. Now, with the highly anticipated launch of the Apple Watch, the screens are getting even smaller. How are smart publishers thinking about the right way to serve users and maintain their attention on smartwatches?
792A wave of distributed content is coming — will publishers sink or swim?
Instead of just publishing to their own websites, news organizations are being asked to publish directly to platforms they don’t control. Is the hunt for readers enough to justify losing some independence?
These stories are our most popular on Twitter over the past 30 days.
See all our most recent pieces ➚
Encyclo is our encyclopedia of the future of news, chronicling the key players in journalism’s evolution.
Here are a few of the entries you’ll find in Encyclo.   Get the full Encyclo ➚
Investigative Reporting Workshop
Voice of San Diego
Daily Mail
Hechinger Report
Hacks/Hackers
St. Louis Globe-Democrat
Charlottesville Tomorrow
EveryBlock
The Times of London
Quartz
Las Vegas Sun
PBS