Nieman Foundation at Harvard
How YouTube’s recommendations pull you away from news
ABOUT                    SUBSCRIBE
June 27, 2019, 6 a.m.

Habit formation: How The Wall Street Journal turned user-level data into a strategy to keep subscribers coming back

The Journal went on a quest to identify the user actions — an app download, an article share, repeat reading of a particular reporter’s stories — that can turn a new subscriber into a loyal one. Then it turned that knowledge into churn-reducing action.

With more than 2.5 million paid members, The Wall Street Journal is the biggest it has ever been. While we will continue to sharpen and evolve our member acquisition strategy, we believe that the next chapter of growth will be fueled by retention. Data clearly shows that the best way to reduce churn is to increase engagement — but the path to driving product use and building loyalty amongst members has not always been as obvious.

Over the past year, a cross-functional group here at the Journal has worked together to identify retention-driving actions and reinvent the way we promote those habits to our member base. We call it Project Habit.

We’ve known for some time that if a member downloads our mobile app or signs up for an email newsletter, they’re more likely to stay with the Journal. Project Habit began when someone said: “There must be more.” Members of our data analytics, optimization, and membership teams set out to confirm that assertion.

Our first step was to make an exhaustive list of all the things a member could do on our site. The list was quite long, including actions from Email Article and Play Puzzle to Build Watchlist and Comment. Next, we built a model to ingest all of these actions or “habits.” We decided to borrow techniques commonly used in medicine by applying the Kaplan-Meier estimator to member retention. Essentially, the model compares the “survival rate” of members who have taken a particular action against the rate of those who did not. It looked at the impact of performing these actions within the first 100 days of membership and how it affected retention in 30-day increments over the course of their first year of tenure.

Why did we only look at actions taken within the first 100 days? To try to eliminate the causation vs. correlation question. Focusing on our newest members ensures we’re looking at those who aren’t yet habituated. Any behavior they display marks the emergence of a new pattern rather than the presence of an existing one. The trends we observed held true for members of all tenures and pack types.

Our big finding: There are many more actions beyond app downloading and newsletter subscribing that drove retention amongst members. We saw, in fact, that there are dozens of habits that clearly reduce churn.

We plotted the actions based on how dramatically they affect retention and their current adoption rate amongst all members. Here are some of the top line trends we found:

  • LOYALTY: Whether it be loyalty to a particular section or a particular journalist, members who consistently visited the same coverage area or author proved to stay longer. The particular area or author didn’t matter — as long as they were loyal to it.
  • CADENCE: Those who read content that publishes with a regular and obvious cadence stayed in at an above-average rate. When readers know exactly when to expect something, they come to rely on it and read it each time it comes out.
  • PLAY: While we’ve known our members rely on the Journal’s financial and political reporting to inform their business decisions and professional lives, the model revealed that lean-back content and features drive retention too.

Transforming our engagement strategy

Armed with the lessons of Project Habit, we were able to make data-driven and meaningful adjustments to our engagement strategy. We used the newfound data to diversify and optimize our existing communications, as well as champion a longstanding need for more prominent member messaging within our products.

A critical insight from Project Habit was that habits are formed early on. If you look at a member’s likelihood to adopt a new habit, that percentage is halved by Day 50 and plateaus soon after Day 100.

With this understanding, we knew it was critical to prioritize onboarding to capitalize on a member’s most impressionable time. We optimized and expanded our onsite onboarding experience to include several new habits. We also extended our email welcome series from 12 days to 100, adding touch points we knew to drive meaningful engagement. Today, 8 out of 10 new members take at least one retention-driving action during onboarding.

At the same time, we partnered with our product, design, and engineering (PDE) team to launch Project Canvas. We now had data to support exactly what we should be promoting to members, but a very limited canvas on which to do it. The Canvas initiative focused on creating more and better spaces to communicate with members. Historically, we had relied almost solely on email as a communication channel. While email remains an important part of many campaigns, the channel’s reach is, at best, limited to those who are opted into marketing comms. We knew it was imperative to build new and effective placements to promote our lengthy list of churn-busting habits.

At the Journal, we always look for the lowest-hanging fruit, and in this case, that was on We grew the number of onsite member messaging placements from 1 to 5. We worked with PDE and the newsroom to ensure the messages looked like a cohesive part of the product and enhanced the member experience without compromising our journalistic or brand standards. Each placement has proven to be at least 2× — and in some cases 55× — more engaging than email.

Project Habit also helped us plan several in-life marketing campaigns around our stickiest content and product features. One campaign, Daily Dose, leveraged the trend revealed about cadence to program a member’s week by assigning a sticky action to each day of the week. In addition to owned media placements, we can now spend confidently in paid channels since we’re able to more precisely understand the value associated with a member action. We found that some of the rarely adopted actions quickly drew the most engagement amongst members. This proved that low adoption didn’t necessarily mean lack of interest, but lack of awareness.

The results

Our data-driven strategy is starting to pay off. Over the past year, we’ve seen active days — our key engagement metric — steadily rise. While there’s no silver bullet to soundly eliminate churn, our quantifiable understanding of what actions drive retention gives us powerful ammunition in the fight against it. Coupling this insight with bigger, better space to communicate with members has allowed us to build an engagement strategy that drives better product usage and retention.

Anne Powell is associate director of engagement, John Wiley is associate director of data science and analytics, and Peter Gray is vice president of product optimization at The Wall Street Journal.

Animation by Romain Loubersanes used under a Creative Commons license.

POSTED     June 27, 2019, 6 a.m.
Show tags
Join the 60,000 who get the freshest future-of-journalism news in our daily email.
How YouTube’s recommendations pull you away from news
Plus: News participation is declining, online and offline; making personal phone calls could help with digital-subscriber churn; and partly automated news videos seem to work with audiences.
Apple brings free call recording and transcription to iPhones; journalists rejoice
“There are decades where nothing happens, and there are weeks when decades happen.”
What can The Wall Street Journal’s new ad campaign tell us about its future?
The new brand campaign is aimed at younger versions of existing Journal readers. The various “It’s Your Business” ads center some of the newsroom’s edgier and more evergreen journalism.