HOME
          
LATEST STORY
Ken Doctor: Why The New York Times hired Kinsey Wilson
ABOUT                    SUBSCRIBE
July 23, 2013, 12:22 p.m.
fuego1

Introducing OpenFuego, your very own heat-seeking Twitter bot

We’re open-sourcing the code behind Fuego, which mines Twitter to find the links a community is talking about most. Build your own!

Editor’s note: Today we’re announcing the release of OpenFuego, an open-source version of Fuego, our popular Twitter robot. Andrew Phelps — formerly of Nieman Lab, now at The New York Times — wrote Fuego’s backend code and has been working to convert it to sharable shape in his spare time. We hope you give it a download and build your own Fuego.

I just want to say that sharing your code with the world for the first time is terrifying. Hold me.

Two years ago, when I was a staff writer at the Lab, boss Josh Benton asked me to build what would become Fuego, a robot that monitors Twitter to surface the stories that journalism nerds are sharing and talking about.

Actually, first he asked me if I knew how to build something like that; I lied and said yes. The idea interested me, and I figured the best way to finally learn how to program would be to tackle a project. I wrote some pretty clunky PHP, the only scripting language I (kind of) knew. I resorted to a lot of hacks and shortcuts, but it worked. And as long as it worked, no one had to know what was under the hood.

Fuego (and its accompanying Twitter account) became a daily habit for a lot of readers and a reliable way for us to keep up with the future-of-news world. The best way we could come up with to describe it was this: It’s like being on Twitter all day long without having to be on Twitter all day long. To this day I receive many, many emails from people asking (a) if I could make a Fuego for their organization or (b) if the Lab would ever open-source it.

fuego2

When Twitter announced it would make significant changes to its API, Fuego needed an overhaul. And now that Twitter has finally flipped the off switch on API v1.0, we decided this would be a good moment to release a version of Fuego you can install and use yourself. In keeping with the open-source spirit of our friends across town, OpenFuego is now available and distributed under an MIT license.

Which brings me to terrifying. I’ve always been a tinkerer but never a programmer. Most people I’ve met who can write code, at least in the journalism world, say they aren’t programmers either. A lot of people seem to learn by accident. But writing code is practically a daily part of my job in the newsroom, even if it’s not in my job description.

So I lay myself bare, my amateurish code made available for public scrutiny, hoping you’ll help make it better and be inspired to do cool new things.

fuego3

A bit about how it works: The setup process is like WordPress. You create a database and follow the instructions in the config file. Then it’s time to play curator. Choose a group of authorities — the Twitter users who will seed the universe you want to track. Say you’re a beat reporter covering the universe of local politics. You might select 10 authorities — a mix of your fellow beat reporters, local politicians, a gadfly or two, and anyone else who’s really engaged in the business of city hall or local government.

(What makes for a good authority? It’s not necessarily that they tweet a lot about the subject you’re interested in — although that helps. What makes an authority good or bad is how well they’ve curated the collection of people they follow. So, in this example, if your local mayor follows a bunch of local politicos and other government officials, she could be a good OpenFuego authority. But if she’s a huge Mets fan and, of the 100 people she follows, 70 are baseball-related, she’s probably not going to be particularly useful to your OpenFuego — even if she’s in the corner office at city hall.)

After you identify these 10 authorities, OpenFuego does the rest. The app follows them on Twitter, as well as all of the people they follow, up to a total of 5,000 sources. When one of those sources shares a link, the link is scored according to the source’s influence. Influence is determined by the number of authorities who follow that source.

This process happens in the background, 24 hours a day. What we’re releasing now, OpenFuego, is just the back-end part. If you want to display that data on a web page — the way we do at niemanlab.org/fuego — you’ll have to make that yourself. (Our implementation of Fuego allows us to look at only stories that have surfaced over a set period of time — the last four hours, the last 24 hours, the last week, and so on — and to rate more recently surfaced stories higher than older ones, to emphasize the newsiness of what it’s finding.) OpenFuego contains a function to make database queries easy, and we’ll be providing some front-end examples to help get you started.

For more details, and to fork the code, see the GitHub project page. Please be gentle.

POSTED     July 23, 2013, 12:22 p.m.
SHARE THIS STORY
   
Show comments  
Show tags
 
Join the 15,000 who get the freshest future-of-journalism news in our daily email.
Ken Doctor: Why The New York Times hired Kinsey Wilson
The former chief content officer at NPR will be moving up I-95 to one of the most important digital positions at the Times.
Why Google is taking another shot at helping readers pay for news
Google Contributor is the latest tool the company has designed to help readers pay for what they read online. But its previous experiments in supporting paid content have had limited success.
In Canada, newspapers’ attempts to experiment with ebooks haven’t seen much success
A number of papers across the country started ebook programs in the early part of this decade, repurposing their archives or producing new work. They haven’t been the moneymakers some had hoped.
What to read next
718
tweets
Ken Doctor: The New York Times’ financials show the transition to digital accelerating
The numbers may look flat, but they contain a continuing set of ups and downs. Up next: executing on a year’s worth of launches.
540Here’s some remarkable new data on the power of chat apps like WhatsApp for sharing news stories
At least in certain contexts, WhatsApp is a truly major traffic driver — bigger even than Facebook. Should there be a WhatsApp button on your news site?
502Controlled chaos: As journalism and documentary film converge in digital, what lessons can they share?
Old and new media types from journalism, documentary, and technology backgrounds gathered at MIT to share practices and discuss mutual concerns.
These stories are our most popular on Twitter over the past 30 days.
See all our most recent pieces ➚
Encyclo is our encyclopedia of the future of news, chronicling the key players in journalism’s evolution.
Here are a few of the entries you’ll find in Encyclo.   Get the full Encyclo ➚
The Fiscal Times
U.S. News & World Report
DNAinfo
Semana
Fox News
El País
Talking Points Memo
The Wall Street Journal
Knight Foundation
NewsTilt
Reuters
Vox Media