Nieman Foundation at Harvard
Thinking about the sheer volume of information — stories, images, videos, data — available from The New York Times can evoke a simultaneous glee and terror. Skimming Times Wire gives you an idea of the hundreds of pieces of content they produce each day. For readers, it’s a tip-of-the-iceberg thing: Yes, on a day-to-day basis you have access to the news and a decades-spanning archive, but you’re not seeing anything close to all of it.
The task (or, more accurately, one of the tasks) for beta620, the Times experimental projects group, is to find a better way to make the newspaper’s information more readily available — both to readers and to the Times itself. Their latest stab at the problem is something they’re calling Deep Dive, a project that aims to give readers a richer, more nuanced understanding of stories.
Deep Dive uses the Times’ massive cache of metadata from stories to go, as the name suggests, deeper into a news event by pulling together related articles. So instead of performing a search yourself within the Times and weeding out off-topic results, Deep Dive would provides readers a collection of stories relating to a topic, based on whatever person, place, event or topic of their choosing. So let’s say you’re interested in protests in Yemen, with Deep Dive you could use an article from nytimes.com as a seed and let the system collect a history of previous items relating to news from the region.
It’s a novel project, still just in demonstration phase — one that aims to let the Times put its extensive archives to better use, but also to create a different experience for consuming news. David Erwin, a software engineer with beta620, said they began building Deep Dive to address problems they were having in following the life of a single news story — and the realization that they were in a position to do something about that. “It kind of came organically from a need that we felt ourselves to better understand what was going on as well as an expression of the potential of in our data and technology,” Erwin said.
What’s interesting about Deep Dive? At least three things:
Deep Dive relies on the extensive tagging system the Times uses for all its stories and makes the Times Topics pages possible. As part of the editing flow tags are applied to stories by editors or producers, with suggestions provided by an internal algorithm. Deep Dive looks for connections among topics, so in the case of our Yemen story, it would likely find other stories on protests in the Middle East.
At the moment Deep Dive is limited to topic tags, which are mostly broad terms like “Middle East and North Africa Unrest (2010- )” and “Demonstrations, Protests, and Riots.” That means that that Yemen story connects with stories in on protests in Egypt and Syria, not more stories about what’s going on in Yemen. Erwin said they hope in the future the system could incorporate other factors to make connections through semantic data, editorial data, or time elements. The Times’ metadata is likely the richest of any news organization.
Part of the challenge will be to figure out what level of specificity readers want in a dive — how related they want their stories to be. In other words, does someone “diving” off a story about Italy’s debt crisis want more stories about Italian politics? The Euro crisis in places like Greece or Spain? The state of the global economy? How do you weigh an older story that’s spot-on versus one that’s breaking now but slightly more off-topic?
“What Deep Dive does it brings you some of the relevance of the topic as you’re reading the article itself,” Marc Frons, the Times CTO for digital operations told me. “So you’re immersed in a topic and you’re going further in. You don’t have to leave the page, which I think is very powerful.”
What Frons is referring to is Deep Dive’ unique interfact, where the related articles flow into the same frame as the main story when selected. You need never leave the page; jumping backwards or forwards in articles all happens in the same space. That’s a departure from the pageview-driven way most news sites are designed. But Deep Dive’s UI matches its underlying thesis: that individual articles are really pieces of a larger story, told in pieces over time and across bylines and datelines.
Frons said the design is just one possible look for the project and not locked in. Still, the design in some ways aligns with Times Skimmer and Times Reader, which may be why Frons sees a future for it. “To me, this sort of notion you have to click and go to another page and wait for the page to load is something that is going to seem very quaint in the not too distant future,” he said. “It’s probably going to seem like using the international operator to place a call to France.”
More interesting, Deep Dive will also allows users to save their “dives,” which would be constantly updated with new articles. While there are plenty of tools that let a person tailor what news they read online, they’re often based around broad categories (like sports or politics), keyword searches (like Times email alerts), or social networks. What Deep Dives promises is an alert more directly based around a specific developing story — story in the “Arab Spring” sense, not story in the one-specific-article sense. Erwin said the idea is to create a personal news experience that will provide contextual information and be consistently updated. “It basically allows you to keep up with your interests over time by providing custom feeds of articles as they are written,” Erwin said.
But beyond those elements, the real promise of Deep Dive, though, is that it continues to show the Times’ flexibility in providing different ways for different kinds of readers to access its content. For some, the blog-style Times Wire, with its constant reverse-chronological updates, will be perfect. For others, a quick skim of the front page of nytimes.com, with its added editorial judgment about what’s most important at the moment, will work better. Grazers and hunters come in a variety of forms, and the Times is trying to better use the enormous amount of work it generates daily to reach as many of them as they can.
“The reader who wants to dive into a topic and better understand an article in the context of a story isn’t really reading the news in the same was as a reader who is browsing current news,” Erwin said. But the audience that wants to pull back the curtain on a story exists, the Times is seeing it, Frons said. There are heavy users — often subscribers — but also others like researchers and professionals who consume a lot of content from the Times, he said. But the lines between heavy users and casual browsers can often blur, and the reader who stops by the Times for a Newt Gingrich story might wind up looking for more context, Frons said.
“What Deep Dive and some of the other things we’re thinking about do is make it almost easier to serve both people at once,” he said.
It could also be a method of converting those browsers into heavy users — making the grazer a hunter. The ability to save dives — or as Erwin says, “putting a pin in a progression of articles along a certain story” — could be useful when you’re trying to follow a story but then miss the latest developments because you didn’t check nytimes.com on a given day. Another thing beta620 would like to explore is the idea of sharing dives among friends, Erwin said.
It may be some time before Deep Dive is ready for the spotlight, both Frons and Erwin say its likely to stay in the experimental stage as they try and refine the product, especially to make its ability to find commonalities between stories more granular. Frons said it’s too soon to know how something like Deep Dive would be affected by the Times paywall, but ideally it would be available to all readers whether or not they’re subscribers. “Now that we’re a paid site we really want to build things that encourage the subscriber to use the site more and encourage people to subscribe by giving them dynamic and innovative features,” he said.
