HOME
          
LATEST STORY
Opening up the archives: JSTOR wants to tie a library to the news
ABOUT                    SUBSCRIBE
July 23, 2013, 12:22 p.m.
fuego1

Introducing OpenFuego, your very own heat-seeking Twitter bot

We’re open-sourcing the code behind Fuego, which mines Twitter to find the links a community is talking about most. Build your own!

Editor’s note: Today we’re announcing the release of OpenFuego, an open-source version of Fuego, our popular Twitter robot. Andrew Phelps — formerly of Nieman Lab, now at The New York Times — wrote Fuego’s backend code and has been working to convert it to sharable shape in his spare time. We hope you give it a download and build your own Fuego.

I just want to say that sharing your code with the world for the first time is terrifying. Hold me.

Two years ago, when I was a staff writer at the Lab, boss Josh Benton asked me to build what would become Fuego, a robot that monitors Twitter to surface the stories that journalism nerds are sharing and talking about.

Actually, first he asked me if I knew how to build something like that; I lied and said yes. The idea interested me, and I figured the best way to finally learn how to program would be to tackle a project. I wrote some pretty clunky PHP, the only scripting language I (kind of) knew. I resorted to a lot of hacks and shortcuts, but it worked. And as long as it worked, no one had to know what was under the hood.

Fuego (and its accompanying Twitter account) became a daily habit for a lot of readers and a reliable way for us to keep up with the future-of-news world. The best way we could come up with to describe it was this: It’s like being on Twitter all day long without having to be on Twitter all day long. To this day I receive many, many emails from people asking (a) if I could make a Fuego for their organization or (b) if the Lab would ever open-source it.

fuego2

When Twitter announced it would make significant changes to its API, Fuego needed an overhaul. And now that Twitter has finally flipped the off switch on API v1.0, we decided this would be a good moment to release a version of Fuego you can install and use yourself. In keeping with the open-source spirit of our friends across town, OpenFuego is now available and distributed under an MIT license.

Which brings me to terrifying. I’ve always been a tinkerer but never a programmer. Most people I’ve met who can write code, at least in the journalism world, say they aren’t programmers either. A lot of people seem to learn by accident. But writing code is practically a daily part of my job in the newsroom, even if it’s not in my job description.

So I lay myself bare, my amateurish code made available for public scrutiny, hoping you’ll help make it better and be inspired to do cool new things.

fuego3

A bit about how it works: The setup process is like WordPress. You create a database and follow the instructions in the config file. Then it’s time to play curator. Choose a group of authorities — the Twitter users who will seed the universe you want to track. Say you’re a beat reporter covering the universe of local politics. You might select 10 authorities — a mix of your fellow beat reporters, local politicians, a gadfly or two, and anyone else who’s really engaged in the business of city hall or local government.

(What makes for a good authority? It’s not necessarily that they tweet a lot about the subject you’re interested in — although that helps. What makes an authority good or bad is how well they’ve curated the collection of people they follow. So, in this example, if your local mayor follows a bunch of local politicos and other government officials, she could be a good OpenFuego authority. But if she’s a huge Mets fan and, of the 100 people she follows, 70 are baseball-related, she’s probably not going to be particularly useful to your OpenFuego — even if she’s in the corner office at city hall.)

After you identify these 10 authorities, OpenFuego does the rest. The app follows them on Twitter, as well as all of the people they follow, up to a total of 5,000 sources. When one of those sources shares a link, the link is scored according to the source’s influence. Influence is determined by the number of authorities who follow that source.

This process happens in the background, 24 hours a day. What we’re releasing now, OpenFuego, is just the back-end part. If you want to display that data on a web page — the way we do at niemanlab.org/fuego — you’ll have to make that yourself. (Our implementation of Fuego allows us to look at only stories that have surfaced over a set period of time — the last four hours, the last 24 hours, the last week, and so on — and to rate more recently surfaced stories higher than older ones, to emphasize the newsiness of what it’s finding.) OpenFuego contains a function to make database queries easy, and we’ll be providing some front-end examples to help get you started.

For more details, and to fork the code, see the GitHub project page. Please be gentle.

POSTED     July 23, 2013, 12:22 p.m.
SHARE THIS STORY
   
Show comments  
Show tags
 
Join the 15,000 who get the freshest future-of-journalism news in our daily email.
Opening up the archives: JSTOR wants to tie a library to the news
Its new site JSTOR Daily highlights interesting research and offers background and context on current events.
Six fresh ideas for news design from a #SNDMakes designathon
New media and legacy media came together at the second weekend-long “hackathon” hosted by the Society for News Design.
Where you get your news depends on where you stand on the issues
A new study by the Pew Research Center examines how Americans’ news consumption habits correlate with where they fall on the political spectrum.
What to read next
1020
tweets
The newsonomics of the millennial moment
The new wave of news startups is aiming at a younger audience. But do legacy media companies have a chance at earning their attention?
803A mixed bag on apps: What The New York Times learned with NYT Opinion and NYT Now
The two apps were part of the paper’s plan to increase digital subscribers through smaller, targeted offerings. Now, with staff cutbacks on the way, one app is being shuttered and the other is being adjusted.
537Watching what happens: The New York Times is making a front-page bet on real-time aggregation
A new homepage feature called “Watching” offers readers a feed of headlines, tweets, and multimedia from around the web.
These stories are our most popular on Twitter over the past 30 days.
See all our most recent pieces ➚
Fuego is our heat-seeking Twitter bot, tracking the links the future-of-journalism crowd is talking about most on Twitter.
Here are a few of the top links Fuego’s currently watching.   Get the full Fuego ➚
Encyclo is our encyclopedia of the future of news, chronicling the key players in journalism’s evolution.
Here are a few of the entries you’ll find in Encyclo.   Get the full Encyclo ➚
U.S. News & World Report
Daily Mail
Press+
Fwix
Detroit Free Press and Detroit News
Foursquare
Voice Media Group
Bloomberg
Kaiser Health News
Upworthy
Franklin Center
Mother Jones