Nieman Foundation at Harvard
HOME
          
LATEST STORY
Axios launches a premium subscription product aimed at the “dealmakers” among us
ABOUT                    SUBSCRIBE
March 1, 2013, 1:19 p.m.
LINK: www.theverge.com  ➚   |   Posted by: Joshua Benton   |   March 1, 2013

Russell Brandom at The Verge has a piece on Common Crawl, “a non-profit foundation dedicated to providing an open repository of web crawl data that can be accessed and analyzed by everyone.” At one extreme, that dataset could be used to build your own local or targeted search engine; at a smaller scale, it could be a boon to data journalists:

For example, web crawl data can be used to spot trends and identify patterns in politics, economics, health, popular culture and many other aspects of life. It provides an immensely rich corpus for scientific research, technological advancement, and innovative new businesses. It is crucial for our information-based society that the web be openly accessible to anyone who desires to utilize it.

Be forewarned: If you think a Hadoop cluster is a kind of Easter candy, this isn’t the weekend hacking project for you. (Here’s an earlier piece from MIT Technology Review.)

Show tags Show comments / Leave a comment
 
Join the 60,000 who get the freshest future-of-journalism news in our daily email.
Axios launches a premium subscription product aimed at the “dealmakers” among us
After a two-week free trial, Axios Pro costs $600/year for one newsletter or $1,800/year for all Pro newsletters. (There’s no monthly option.)
A new report shows the impact of racial justice protests in 2020 on three local newspapers
A study of crime reporting in three major U.S. dailies found coverage included less dehumanizing language by the end of the year.
Does having stronger local newspapers make people more likely to follow COVID safety guidelines? Er, not so much
A new study finds that the more local newspapers there were in a county, the worse it performed on a measure of social distancing in the early days of the pandemic. But take the findings with a grain of salt.