Nieman Foundation at Harvard
HOME
          
LATEST STORY
How The Washington Post built — and will be building on — its “Knowledge Map” feature
ABOUT                    SUBSCRIBE
May 20, 2009, 11:14 p.m.

The golden age of computer-assisted reporting is at hand

Computer-assisted reporting or CAR has been around, well — ever since there were computers. Even when I was in journalism school (which was longer ago than I care to remember), we learned about databases we could search, etc. But the explosion of Web-based tools and ways of sifting through and sharing data has created something approaching a revolution, and the potential benefits for journalism are only just beginning to reveal themselves. If this movement has a patron saint, it is probably Adrian Holovaty, who gained renown for creating the amazing Chicagocrime.org — one of the first Google Maps mashups — and then worked on data-driven features at the Washington Post, followed by his fellowship-financed Everyblock, which aggregates local data about an area.

Another recent example of how data can drive reporting, and how Web-based tools can extend and enhance that reporting, comes from several British newspapers — primarily The Guardian — and their coverage of an emerging expense scandal involving British politicians. One of the really interesting things that The Guardian has done is to publish all of the expense info they have through a laboriously detailed and publicly accessible Google spreadsheet. As Paul Bradshaw points out at the Online Journalism Blog, this structure actually allows reporters (or in fact anyone who is interested in the info) to extract useful data simply by changing the URL. Someone has even created a page where you can run queries on the database with a simple click.

There are any number of tools out there that can take the data you get from spreadsheets or databases and do useful things with it, such as organizing it into charts the way Many Eyes Wikified (a spinoff from IBM’s Many Eyes) does. Another source of interesting data-driven mini-apps is Yahoo Pipes, an often-overlooked service that lets you create data mashups of various kinds. I’ve already come across pipes that someone created to map your Twitter followers and strip-mine your Twitter stream for links, and I’m sure there are dozens of others. They are relatively easy to create and can be easily customized to do a variety of things.

Is mapping your Twitter followers journalism? Not really. But these tools can be used for all kinds of journalistic efforts, as The Guardian and others have shown. As Holovaty continually points out, we are just scratching the surface of what is possible with the data underlying much of journalism — data that would be a lot easier to remix and mashup and display in different and interesting ways if newspapers identified and tagged and indexed that data when stories were being written, instead of trying to do those things retroactively. When the data that is already being collected is freed up, projects like this (a Holovaty production) all of a sudden start to become not just possible but almost easy to generate.

The Guardian, not surprisingly, is pretty far out in front on this — along with the New York Times, which has also been doing a lot of interesting data-driven things. But while the NYT has an open API for stories and data, only The Guardian offers a *full* API of all the content they publish, as well as a “data store” filled with lots of the data they have accumulated on a whole range of stories (if you’re interested in some tips, there’s a great interview here with Tony “the Data Juggler” Hirst,” one of the most active users of The Guardian’s data and APIs). If you’ve got any other great examples of newspapers using data to enhance their journalism, or any useful sites or recommended Yahoo Pipes, please leave links in the comments.

Bonus link:

See Adrian Holovaty’s definitive, two-part answer to the question “is data journalism?”

POSTED     May 20, 2009, 11:14 p.m.
SHARE THIS STORY
   
Show comments  
Show tags
 
Join the 15,000 who get the freshest future-of-journalism news in our daily email.
How The Washington Post built — and will be building on — its “Knowledge Map” feature
The Post is looking to create a database of “supplements” — categorized pieces of text and graphics that help give context around complicated news topics — and add it as a contextual layer across lots of different Post stories.
How 7 news organizations are using Slack to work better and differently
Here’s how Fusion, Vox, Quartz, Slate, the AP, The Times of London, and Thought Catalog are using Slack for workflow — and which features they wish the platform would add.
The New York Times built a robot to help make article tagging easier
Developed by the Times R&D lab, the Editor tool scans text to suggest article tags in real time. But the automatic tagging system won’t be moving into the newsroom soon.
What to read next
1119
tweets
New Pew data: More Americans are getting news on Facebook and Twitter
A new study from the Pew Research Center and Knight Foundation finds that more Americans of all ages, races, genders, education levels, and incomes are using Twitter and Facebook to consume news.
701Newsonomics: The halving of America’s daily newsrooms
If you’re lucky enough to have the right deep-pocketed owner buy your paper and steady it, you’ve won the lottery. If you’re in a town whose paper is owned by the better chains, or committed local ownership, your loss will probably be mitigated. Otherwise, you’re out of luck.
575How 7 news organizations are using Slack to work better and differently
Here’s how Fusion, Vox, Quartz, Slate, the AP, The Times of London, and Thought Catalog are using Slack for workflow — and which features they wish the platform would add.
These stories are our most popular on Twitter over the past 30 days.
See all our most recent pieces ➚
Encyclo is our encyclopedia of the future of news, chronicling the key players in journalism’s evolution.
Here are a few of the entries you’ll find in Encyclo.   Get the full Encyclo ➚
Wikipedia
Wisconsin Center for Investigative Journalism
Demand Media
Tucson Citizen
Spot.Us
Grist
E.W. Scripps
Chi-Town Daily News
San Diego News Network
El País
PubliCola
INDenverTimes