Nieman Foundation at Harvard
If the Philadelphia newspapers wanted to convert to nonprofits, what would stand in their way?
ABOUT                    SUBSCRIBE
July 19, 2012, 10:38 a.m.
Twitter preserved

That plan to archive every tweet in the Library of Congress? Definitely still happening

It has turned out to be quite an undertaking, but the Library plans to make good on its promise to America.

Twitter preserved

A little more than two years ago, the Library of Congress announced it would preserve every public tweet, ever, for future generations.

That’s right. Every public tweet, ever, since Twitter’s inception in March 2006, will be archived digitally at the Library of Congress. That’s a LOT of tweets, by the way: Twitter processes more than 50 million tweets every day, with the total numbering in the billions.

Fifty million tweets a day. How cute. That number is now 400 million, according to Twitter CEO Dick Costolo. (The first comment on the project’s FAQ page sums up much of the Internet’s reaction: “It’s critical the future generations know what flavor burrito I had for lunch.”)

We hadn’t heard about this project in some time. Last week a story on quoted a social-media researcher as saying the LoC “has quietly backed away from the commitment.”

False, said Library spokeswoman Jennifer Gavin; the project is very much still happening. Good librarianship, she said, moves more slowly than Twitter.

“The process of how to serve it out to researchers is still being worked out, but we’re getting a lot of closer,” Gavin told me. “I couldn’t give you a date specific of when we’ll be ready to make the announcement.”

The Library first revealed its plans in a tweet on April 14, 2010, but apparently that was before sorting out with Twitter the logistics of acquiring all that data. Petabytes of data.

“We began receiving the material, portions of it, last year. We got that system down. Now we’re getting it almost daily,” Gavin said. “And of course, as I think is obvious to anyone who follows Twitter, it has ended up being a very large amount of material.”

Gavin said the archive will be made available to anyone with a library card, but only on the premises in Washington. “My understanding is that at this time we do not intend to make it available by web,” she said, but that may be subject to change. It’s not meant to be the Ultimate Twitter Search Box we’ve always dreamed of.

In fact, there will be a six-month embargo on fresh tweets (even though, obviously, the data is publicly available — if you can find it). That agreement has been in place since the deal was struck. Twitter said then the tweets could be used only “for internal library use, for non-commercial research, public display by the library itself, and preservation.”

The challenge now is finding ways to refine the raw data in useful ways. Sort by keywords? Date? Sentiment? Burrito flavor? Gavin said the Library is still figuring out the user interface.

POSTED     July 19, 2012, 10:38 a.m.
Show comments  
Show tags
Join the 15,000 who get the freshest future-of-journalism news in our daily email.
If the Philadelphia newspapers wanted to convert to nonprofits, what would stand in their way?
Some sort of attachment with Temple University could be in the works. But nonprofit law could throw up a number of obstacles to making that happen.
Medium partners with publications like The Awl and Fusion, and more native ads are on the way
Medium is rolling out a slew of changes, including publisher partnerships, updated apps, and plans for advertising.
4 takeaways from The New York Times’ new digital strategy memo
With a renewed focus on subscriptions, the Times believes it can double its digital revenue to $800 million in 2020.
What to read next
What happened after 7 news sites got rid of reader comments
Recode, Reuters, Popular Science, The Week, Mic, The Verge, and USA Today’s FTW have all shut off reader comments in the past year. Here’s how they’re all using social media to encourage reader discussion.
699Facebook woos journalists with Signal, a dashboard to gather news across Facebook and Instagram
Signal helps journalists find, source, and embed content from Facebook and Instagram.
672Get AMP’d: Here’s what publishers need to know about Google’s new plan to speed up your website
The speed gains are very real. But do publishers want to trade in the open space of what we’ve known as the web for yet another platform they have little control over?
These stories are our most popular on Twitter over the past 30 days.
See all our most recent pieces ➚
Encyclo is our encyclopedia of the future of news, chronicling the key players in journalism’s evolution.
Here are a few of the entries you’ll find in Encyclo.   Get the full Encyclo ➚
St. Louis Beacon
Mother Jones
Chicago News Cooperative
International Consortium of Investigative Journalists
New Haven Independent
Ann Arbor News
MediaNews Group