Nieman Foundation at Harvard
HOME
          
LATEST STORY
The California Journalism Preservation Act would do more harm than good. Here’s how the state might better help news
ABOUT                    SUBSCRIBE
March 1, 2013, 1:19 p.m.
LINK: www.theverge.com  ➚   |   Posted by: Joshua Benton   |   March 1, 2013

Russell Brandom at The Verge has a piece on Common Crawl, “a non-profit foundation dedicated to providing an open repository of web crawl data that can be accessed and analyzed by everyone.” At one extreme, that dataset could be used to build your own local or targeted search engine; at a smaller scale, it could be a boon to data journalists:

For example, web crawl data can be used to spot trends and identify patterns in politics, economics, health, popular culture and many other aspects of life. It provides an immensely rich corpus for scientific research, technological advancement, and innovative new businesses. It is crucial for our information-based society that the web be openly accessible to anyone who desires to utilize it.

Be forewarned: If you think a Hadoop cluster is a kind of Easter candy, this isn’t the weekend hacking project for you. (Here’s an earlier piece from MIT Technology Review.)

Show tags
 
Join the 60,000 who get the freshest future-of-journalism news in our daily email.
The California Journalism Preservation Act would do more harm than good. Here’s how the state might better help news
“If there are resources to be put to work, we must ask where those resources should come from, who should receive them, and on what basis they should be distributed.”
“Fake news” legislation risks doing more harm than good amid a record number of elections in 2024
“Whether intentional or not, the legislation we examined created potential opportunities to diminish opposing voices and decrease media freedom — both of which are particularly important in countries holding elections.”
Dateline Totality: How local news outlets in the eclipse’s path are covering the covering
“Celestial events tend to draw highly engaged audiences, and this one is no exception.”