Twitter  Questions to ask when you're planning a local news event nie.mn/1p8mDLr  
Nieman Journalism Lab
Pushing to the future of journalism — A project of the Nieman Foundation at Harvard

The golden age of computer-assisted reporting is at hand

Computer-assisted reporting or CAR has been around, well — ever since there were computers. Even when I was in journalism school (which was longer ago than I care to remember), we learned about databases we could search, etc. But the explosion of Web-based tools and ways of sifting through and sharing data has created something approaching a revolution, and the potential benefits for journalism are only just beginning to reveal themselves. If this movement has a patron saint, it is probably Adrian Holovaty, who gained renown for creating the amazing Chicagocrime.org — one of the first Google Maps mashups — and then worked on data-driven features at the Washington Post, followed by his fellowship-financed Everyblock, which aggregates local data about an area.

Another recent example of how data can drive reporting, and how Web-based tools can extend and enhance that reporting, comes from several British newspapers — primarily The Guardian — and their coverage of an emerging expense scandal involving British politicians. One of the really interesting things that The Guardian has done is to publish all of the expense info they have through a laboriously detailed and publicly accessible Google spreadsheet. As Paul Bradshaw points out at the Online Journalism Blog, this structure actually allows reporters (or in fact anyone who is interested in the info) to extract useful data simply by changing the URL. Someone has even created a page where you can run queries on the database with a simple click.

There are any number of tools out there that can take the data you get from spreadsheets or databases and do useful things with it, such as organizing it into charts the way Many Eyes Wikified (a spinoff from IBM’s Many Eyes) does. Another source of interesting data-driven mini-apps is Yahoo Pipes, an often-overlooked service that lets you create data mashups of various kinds. I’ve already come across pipes that someone created to map your Twitter followers and strip-mine your Twitter stream for links, and I’m sure there are dozens of others. They are relatively easy to create and can be easily customized to do a variety of things.

Is mapping your Twitter followers journalism? Not really. But these tools can be used for all kinds of journalistic efforts, as The Guardian and others have shown. As Holovaty continually points out, we are just scratching the surface of what is possible with the data underlying much of journalism — data that would be a lot easier to remix and mashup and display in different and interesting ways if newspapers identified and tagged and indexed that data when stories were being written, instead of trying to do those things retroactively. When the data that is already being collected is freed up, projects like this (a Holovaty production) all of a sudden start to become not just possible but almost easy to generate.

The Guardian, not surprisingly, is pretty far out in front on this — along with the New York Times, which has also been doing a lot of interesting data-driven things. But while the NYT has an open API for stories and data, only The Guardian offers a *full* API of all the content they publish, as well as a “data store” filled with lots of the data they have accumulated on a whole range of stories (if you’re interested in some tips, there’s a great interview here with Tony “the Data Juggler” Hirst,” one of the most active users of The Guardian’s data and APIs). If you’ve got any other great examples of newspapers using data to enhance their journalism, or any useful sites or recommended Yahoo Pipes, please leave links in the comments.

Bonus link:

See Adrian Holovaty’s definitive, two-part answer to the question “is data journalism?”

                                   
What to read next
org-chart-cc
Ken Doctor    Aug. 25, 2014
“Things” editor, distribution editor, correspondent for progress — as newsrooms change, so do the ways they organize their human resources.