Twitter  Quartz found an unlikely inspiration for its relaunched homepage: The email newsletter. nie.mn/1AQXuxD  
Nieman Journalism Lab
Pushing to the future of journalism — A project of the Nieman Foundation at Harvard

Jennifer 8. Lee on raw data, APIs, and the growth of “Little Brother”

Editor’s Note: We’re wrapping up 2010 by asking some of the smartest people in journalism what the new year will bring.

Here, Jennifer 8. Lee gives us predictions, about the growing role of raw data, the importance of APIs, and the need for a break-out civic mobile app.

Raw data and the rise of “Little Brother”

In 2011 there will be a slew of riffs on the WikiLeaks anonymous dropbox scheme, sans gender drama — at least one of them by former WikiLeakers themselves. It will remain to be seen how protective the technologies are.

Basically, this codifies the rise of primary source materials — documents, video, photos — as cohesive units of consumable journalism. Turns out, despite the great push for citizen journalism, citizens are not, on average, great at “journalism.” But they are excellent conduits for raw material — those documents, videos, or photos. They record events digitally as an eyewitness, obtain documents through Freedom of Information requests, or have access to files through the work they do. We are seeing an important element of accountability journalism emerge.

Big Brother has long been raised as a threat of technological advancement (and certainly the National Security Agency has done its fair share of snooping). But in reality, it is the encroachment of Little Brother that average Americans are more likely to feel in our day-to-day lives — that people around us carry digital devices that can be pulled out for photo or videos, or they can easily copy digital files (compared to the months of covert photocopying that Ellsberg did for 7,000 pages) that others would rather not have shared with the world.

One notable strength of raw material is that it has a natural viral lift for two reasons: audience engagement, and the way legacy media operates with regard to sourcing and competition. Social media is a three-legged stool: create, consume, and share content. Because original material often feels more like an original discovery, it is more appealing to share. Documents, videos, and photos are there for anyone to examine and experience firsthand. The audience can interpret, debate, comment as they choose, and they feel greater freedom to reupload and remix that material, especially video.

The importance of APIs

There will also be an explosion in shift from raw data to information made available by application programming interfaces. A good example is ScraperWiki, out of the United Kingdom, which scrapes government data into repositories and then makes it available in an API.

Government agencies are hearing the public cry for data, and they are making raw data available. Sometimes it’s in friendlier formats like .csv or .xls. Sometimes it is in less usable formats, like PDF (as the House of Representatives did with a 3,000-page PDF of expenses) and even .exe files. (As the Coast Guard’s National Response Center has done with its incident data. It’s an extractable .xls with a readme. I know. It makes a lot of people cringe. At least their site isn’t also in Flash.) As part of this open push, the Obama administration has set up data.gov.

As that comes out, people are realizing that it’s not enough to get the public to bite, even though the underlying data might contain interesting material. It needs to be even easier to access. A good example of what happens when something becomes easily searchable: ProPublica‘s Dollars for Docs project, on payments doctors received from pharmaceutical companies, generated an explosion of interest/investigations by taking data that was already technically public and standardizing it to make it searchable on the Internet.

What we need: the great civic mobile app

What we’re still waiting for: The break-out civic mobile app, a combination of Craigslist and Foursquare, where a critical mass of people can “check in” with comments, photos and complaints about their local community. It’s unclear how this will happen. Perhaps it will be built on the geolocation tools offered by Facebook or Twitter. Perhaps it will be an extension of Craigslist, which already has a brand associated with local community. Perhaps it’ll be something like SeeClickFix, which allows people to register complaint about potholes or graffiti, or CitySeed, a mobile app the Knight Foundation has given a grant to develop.

[Disclosure: Both the Knight Foundation and Lee are financial supporters of the Lab.]

                                   
What to read next
ferguson-protest-night-ap
Mark Coddington    Aug. 22, 2014
Plus: Controversy at Time Inc., more plagiarism allegations, and the rest of the week’s journalism and tech news.
  • Ryan

    See, Click, Fix. Seeclickfix.com SeeClickFix allows anyone to report and track non-emergency issues anywhere in the world via the internet. This empowers citizens, community groups, media organizations and governments to take care of and improve their neighborhoods.

  • http://www.oliviertravers.com/ Olivier Travers

    A favorite passive aggressive tactic of government agencies that clearly don’t like being forced into disclosure is to publish “scanned” PDFs where text is just a big image. This of course makes it impossible to index and search these documents, and if you want to quote, you have to retype the content yourself.

  • Pingback: links for 2010-12-22 « Blarney Fellow

  • http://www.richardcornish.com Rich

    Jennifer, great perspective on things to come. Don’t you think that geo targeting might be an aspect of the API rather than thinking of localization as a first stop? I suppose it all depends on your scope.

    I know former Knight Grant recipient EveryBlock is tackling this problem on several fronts: working with governments, organizing and geocoding the data, and then recently providing an API for it. Their Discussions are also starting to get traffic, where citizens have started using it as an outlet to voice public concerns. I think it’s definitely one to watch.

  • Pingback: Primary Source Journalism and the Rise of “Little Brother” | Jennifer 8. Lee

  • http://groundcrew.us Joe Edelman

    I think “the great civic mobile app” you’re referring to is Groundcrew (http://groundcrew.us)… also a Knight funded project. I’m CEO.

  • alyena khan

    The relevant term of art here is walled gardens  Christmas gifts for brother the proprietary data
    of companies like facebook, which are ultimately unaccountable to the
    public at large, is not available for every and all citizen to access,
    remix, and reuse in their personal communications Christmas gifts for boyfriend.

  • alyena khan

    I agree that for programming, multi-monitors is
    great Birthday Messages. For web programming, I have one monitor rotated  degrees for
    viewing source code more lines per screen), and the other at a normal
    angle for viewing web pages as the customer sees them  Short Birthday Poems

  • http://www.squidoo.com/lensmasters/bueler Bueler

    Since the publication of this post, has a civic mobile app been developed?