Nieman Foundation at Harvard
When we at the Lenfest Local Lab introduced our second experiment, we started with this observation about local news:
Oftentimes local news stories are written about the places we live, work or pass by — but we would never know it because there’s no front page in the physical world. Addressing this issue, and breathing geographic life into local coverage, was the inspiration for our first experimental local news app.
Our way of exploring the issue was to test an app that sends people local news stories about where they actually are. Unlike national or international newsrooms that are incentivized to send or organize stories by continent, country, state, or city — local newsrooms can organize stories about neighborhoods, wards, counties, and towns. They write about block-level issues, and if technology lets us deliver stories by block now, the chances that a nearby local story will be relevant to someone increases significantly.
A few months before launching the app we approached The Philadelphia Inquirer about collaborating on a location-aware app. We envisioned an app that would know where someone was throughout the day, and if that person passed by a place the Inquirer wrote about, they would receive a notification. Effectively, we wanted to test turning streets in Philadelphia into a personal local news homepage.
We had Inga Saffron’s architecture coverage in mind. Inga is a Pulitzer Prize-winning journalist who has covered the region’s architecture, planning, and development for nearly two decades. Her coverage was a good starting point because her stories tend to be relevant for a long time, and we liked the idea that something other than the endless news cycle would drive people to use the app. Instead, their actual pathways through the city would set off news notifications about where they were, block by block. This wasn’t going to be a news walking tour — it was going to be a news tour of where they were walking.
After having a few more conversations with the Inquirer and with Inga, we decided to add some public art and real estate stories to the app. Art and real estate content were a natural fit alongside architecture, and including them would expand the size of our potential audience for the experiment.
So now we had an even bigger pool of high-quality evergreen local stories to pull into the app. They wouldn’t need frequent editing or updating, and the stories were tied to a single place, which made geotagging and testing relatively easy. We also thought sending people notifications about buildings they could look up and see would make the notifications easier to understand.
In the future, we could imagine this method working well to surface other types of local news stories, including important local coverage about school safety — like the Inquirer’s Toxic City investigative report, paired with a full safety report for the school nearest to you.
We could also imagine notifying people about an Inquirer business story about a popular hoagie debut. (Yup, we call them hoagies here.) The same goes for a story about a new musical for children opening at a local theater. Parents walking by the Arden Theatre in Old City might want that story to find them, rather than the other way around.
Once you think about all of the possibilities, though, you remember why this is just an early experiment. We’re still trying to test and measure how this new experience could be the most valuable for local newsrooms and local communities. As we mentioned in our launch post about the app:
…geotagging every local news story and notifying people wherever they go is a big, sprawling, and possibly chaotic idea. That’s why we’re starting small and measuring outcomes first before potentially expanding the concept.
Until we know more about when and where people like receiving stories in this way, we’ll keep taking small steps towards what we think is a much bigger idea.
People’s first experience with the app is the series of welcome screens above, asking for user permissions that will make the app work. We ask users to allow access to their location and also to detect their motion so we can skip notifications while they’re driving. We don’t want the app to distract people’s attention from the road, and we want to make it easy for people to stop and engage with the notification and the story.
As we point out later in this post, stopping to read a full article when you’re walking in the city might not be the ideal experience for everyone, and we’re aiming to test new formats in our next iterations of location-aware product experiments.
To get stories into the app, we manually geotagged articles inside our custom open-source CMS. The top three stories that have generated the most notifications for users so far are:
All of these stories are clustered geographically around Love Park, a public plaza adjacent to City Hall. The stories are about a mix of topics, including a critique of the park’s recent overhaul, a story about the renovation of a former bank building, and a profile of a local muralist.
You can read more about the thinking behind the app in our intro post and get a peek into our research and design process here.
The underlying notion for this experiment was the lab’s strong belief that leveraging location information is where local news has a unique advantage to test out new and innovative product experiences. We also believe that incorporating basic principles of location into local news products will benefit many stories and many audiences down the line. But this app was our first test of that idea, and there is more to learn.
Knowing that our plan is to hopefully expand beyond arts, real estate, and architecture content, and to hopefully launch experiments in other local markets, the lab launched the HERE app in mid-November. We conducted our first round of analysis two months later in mid-January, and recently took another look at the engagement patterns to see how the app has been performing so far in 2019.
All of what we did and learned is below, which begins to answer two key questions: Do people want to receive local news about the places they actually are? And should newsrooms invest in the resources and tools to deliver it this way?
Since this was just an experiment, promotion was light. The Inquirer published an article on Philly.com about the app and it was tweeted it out to a few groups we thought would be interested in trying it out, including Philly.com readers (Philly.com is The Philadelphia Inquirer’s website), Inga Saffron’s followers (her journalism provides the foundation for the app) and followers of the Lenfest Lab.
Here are all of the promotional materials:
Philly.com: “Help us test a new way to read and learn about Philly”.
We’re testing a new way to read and learn about Philly. A new app, called HERE For Local Journalism, is designed to help you easily discover information about the city’s architecture, arts scene, and real estate development. Here's how it works ➡️ https://t.co/3LHCj2L2Nh pic.twitter.com/tmTpzjRiJk
— Philly.com (@phillydotcom) December 12, 2018
My innovative colleagues at the @lenfestlab have come up with a cool new app that lets you access my architecture pieces whenever you walk by a building/place I’ve reviewed. Try it, it’s free! https://t.co/16tmbZ00c3
— Inga Saffron (@IngaSaffron) December 27, 2018
Our app is HERE – and live! @schmalie gives an overview of our first experimental app: HERE for Local Journalism. It uses geotagging and motion detection to deliver news when you’re nearby (but not when you’re driving). https://t.co/qJdW0OPeAA
— Lenfest Local Lab (@lenfestlab) November 20, 2018
The app is still available in the App Store, and 355 people have installed it so far. Still, only about a third of them have received an alert so far. Here are a few reasons why:
Graphic by Faye Teng.
The engagement rate for HERE app notifications peaked at 14 percent; an average news alert gets a 3.4 percent engagement rate. This starts to show us that paying attention to where people are and what they’re interested in can pay off when it comes to engagement. The most common news alert — breaking news — definitely appeals to a lot of readers and can drive immediate traffic, but curated alerts for people based on what they like reading about where they have the potential to drive even more meaningful experiences with local news coverage.
Charts by Hero Digital.
The data is directional given the size of the group, but it still shows a 4× increase in engagement over less customized, less location-aware notifications. If newsrooms applied this approach to more local stories, audiences might start relying on them to get trustworthy information about what’s going on around them in real time.
The engagement rate for notifications was 10 percent on weekdays and 14 percent on weekends, with most clicks coming on Sundays. Proof that people have more time to read news on weekends isn’t a new insight, but it’s a useful reminder that even though technology allows us to reach people wherever they are, whenever they’re there — newsrooms should still think critically about the weekday vs. weekend habits of their audiences and build new location-based products that are adaptable for different contexts.
Location-aware features quickly put stories in the hands of people nearby and let evergreen content become relevant again in the places where people go. It opens up a whole new set of possibilities for digital news delivery unconstrained by screen dimensions or third-party algorithms.
At the same time, it would take resources to get it right. We saw that if location-aware notifications were sent for many more stories, an organization would probably need to see enough value in it to assign dedicated editorial resources. Editors would determine whether a reader’s proximity to a story location actually meant it’s more relevant to them. An example would be this story in the app about a grocery store opening.
An editor could decide that this story is only relevant to someone who lives in that neighborhood and not someone passing through. But they also might want to send the notification to a non-resident because they’re interested in commercial real estate news. If newsrooms could select or combine those types of audience characteristics in their own content management systems, rather than select segments through other platforms or tools, they could regain control over the end-to-end local news distribution funnel as well as many related business models.
We were glad to see hundreds of people install the app and also give it permission to access their location and motion. We attribute much of this behavior to the trust people already had in Inga Saffron and the Inquirer and was supported by the transparent language written into our welcome screens. We were thoughtful about the clarity and accuracy of words we used to explain the experience to people, and we were happy to see that 94 percent of people let the app send notifications; 98% allowed it to access their location and 73 percent of people let the app detect whether or not they were driving.
Graphic by Faye Teng.
Directly following a tweet from Inga, app installs doubled from 150 to 300.
We were impressed by the impact of Inga’s tweet for two reasons. First, because it seemed to cause a spike in downloads, but also because it kicked off an online conversation about the app with other local journalists, city employees, and architecture fans. For example, this tweet confirmed that the audience saw how valuable the app could be in terms of helping people feel more connected to the city through local news:
“On the Block Alerts”-I love it! What a great initiative to inform residents and tourists alike about Philly’s built environment. Doubles as a tool to endear passers-by to the idiosyncrasies of the city, and thus feel more connected to it.
— Coby (@Cobylefko) December 28, 2018
This app inspired the creation of a new CMS field: radius. We used a story’s radius to set the geofence for its notification. (For more on how the CMS was built and instructions for using it yourself, read this.) Setting the radius field each time let us decide how close someone should be to receive a story notification.
For stories about narrow buildings on a city block, we set the radius to the closest distance that tested well, 100 meters. That distance was the default range if we didn’t change it, and in the context of an experiment setting the radius was easy. However at scale, this would add a new dimension to the editing process, and someone would need to determine how close a reader should be to a story to receive a notification.
For example, we set the radius wider to 300 meters for this story about Amtrak’s decision to replace the iconic “clickety-clack” flipboard at Philadelphia’s 30th Street Station.
This story also taught us something significant about how much editing the app would need over time. We pinned a story at 30th Street Station in December that wasn’t as relevant later in January and should have been updated. The full timeline of possible stories that could have been ‘pinned’ to the station goes like this:
The lesson came a few days later on January 31, when I received a screenshot of a HERE app notification from my former Philly.com colleague Dave Merrell. Dave works at The Washington Post and was passing through 30th Street Station on his way to New York:
He wanted to share news that the app was working well, but I quickly noticed that the story notification he received was out of date and needed updating.
Even though much of Inga’s coverage is evergreen, some of it relates to developing stories like this one and so an editor would need to make sure that the latest, or best news is attached to each location.
We had ongoing conversations in the lab about how we might apply different user experience rules if people were in the city, the suburbs or in a rural area. In the city, you might want to send a notification to someone who comes within 100 yards of a story location, but in the suburbs, you might want to send a notification to people driving within five miles of this Baptist compound in King of Prussia, Pennsylvania.
Technology makes all of these new types of story delivery possible now, and we can even send notifications silently if we think they’ll be too distracting in the moment. The key will be paying enough attention to these distinct user experiences to start testing the right new rules.
Unlike a one-time news event like an election or the Super Bowl, you can keep these types of evergreen, location-aware products available for a while and see how behavior changes over time. We’re keeping HERE in the App Store for now and will update it with new stories when we can. We’ll keep an eye on it to see if engagement changes at all in the spring when more people will be outside exploring the city. If it does, we’ll be sure to share what we learn.
With this experiment, we just scratched the surface of how newsrooms might organize a locations database to accelerate location-based product development. For local architecture content, tagging a single location was relatively easy — but for less straightforward stories there may be many locations associated with the narrative or none at all. All of those decisions would need to be carefully considered by experienced editors and reporters and data would need to be stored in a useable way.
We were clear in the app’s welcome messaging that we would gather location data a few times a day to help news find you where you were. We gathered this data because we wanted to focus our lab’s limited resources on including stories in the app that were about places people walked most often. We found that people who installed the app were often walking up Market Street or JFK Boulevard between City Hall and 30th Street Station and we did our best to pull in stories about those areas.
A view of background location tracking information for HERE app users.
The information was also good to gather since it’s just one early data point showing how some Inquirer readers move through the city and when. Local newsrooms could use this information to help refine ideas for experiments or new product ideas.
Get more qualitative feedback: For past experiments, the lab has gotten qualitative feedback by sending a survey link out in a notification or a text message. This time only a few people filled out the survey, so next time we’ll do two things differently:
Two options for an in-app user feedback survey by Faye Teng.
Further optimize content for being on-the-go: Some qualitative feedback we received internally was about the tension between walking by a building and stopping to read an article. Next time we’re going to explore new content formats that are easier for reading on-the-go like shorter pieces, highlights or audio.
Consider sending an email instead of an alert: To additionally ease friction between walking and reading news, we’re thinking we might email stories to people about places they’ve been nearby so they can read them later when they have time.
Think about the representativeness of our results: Many local news organizations today are focusing their efforts on big strategic wins. Meanwhile, the insights from our smaller experiments can leave organizations wanting more confidence that the results would scale to a larger audience. As a result, we’ll be taking a closer look at the sample size of our experiments and do our best to make sure they’re as representative of broader audiences as possible.
Analyze potential business models and let the way people use the product direct the conversation around revenue: One reason to run experiments is to quickly discover new products that audiences truly value before building them at scale, which is sometimes called finding a product-market fit before figuring out how to make money. The experiments we run in the lab are intended to find local news product-market fit, which means we’re testing the value of a business model before moving onto the revenue model.
This idea came up in Ken Doctor’s interview with David Perpich, the general manager and president of Wirecutter and former senior vice president of product for The New York Times. Perpich said these two important things we think new product teams should keep in mind:One of the lessons that the Times learned, and I certainly learned with NYT Now, was that we tried to solve product-market fit and monetization at the same time.
The most important thing is that we create something that people love, and then we can get to a scale that would be meaningful enough to make the investment you need to be able to do monetization right.
Local news revenue models today are under a lot of pressure, but in our lab, we focus on what we can control: the quality of the user experience. At the same time, we’ve found that it’s important to sketch out potential business and revenue models for ourselves and our collaborators while we’re building the experiment. We think this will help give stakeholders the information they need to understand how scaling a successful experiment would benefit a bottom line.
For our next experiment, we’re researching a handful of business models that we’ll review a few weeks or months after launch, to see if the way people are using the app supports any of them in particular. Here are some models we’re looking into:
Our next experiment will focus on making local professional restaurant reviews easier to search, browse and save for quicker and better decision making—whether people are long term planning or are on-the-go. It will also incorporate features from the HERE app, letting you know when you’re near a restaurant that you’ve saved as a favorite. We’re looking forward to building on these insights and continuing to explore location-based opportunities for new local news products. Let us know what you think!
A version of this piece appeared on Medium. The Lenfest Local Lab, a small, multidisciplinary product and user experience innovation team located in Philadelphia, is supported by the Lenfest Institute for Journalism and led by the author, Sarah Schmalbach. Here’s her analysis of an earlier experiment.
