Nieman Foundation at Harvard
The New York Times’ new Slack 2016 election bot sends readers’ questions straight to the newsroom
ABOUT                    SUBSCRIBE
Sept. 23, 2013, 11:59 a.m.

The Hub of the Twitterverse: The Boston Globe has built a localized, tweet-powered news aggregator

61Fresh is the latest experiment from Globe Lab, the paper’s R&D department, to find ways to harness social signals to see what readers are talking about.

61freshscreenshotIn the Boston area, the passing of a Dunkin Donuts franchise is news. So are the finer points of Tom Brady’s hair. But in this instance, we know the closing of the Mass Ave. Dunkin reached a certain level of…prominence? virality? social lift? — because it made an appearance on 61Fresh, the experimental news aggregator built by the Globe Lab, the research and development team inside The Boston Globe.

This tweet, in particular, caught fire:

(In fairness, Matt Damon, a napkin, and apples also made that Dunkin newsworthy.)

What, exactly, is 61Fresh? And what’s with the name?

“Half the people who respond to it ask, ‘What’s up with the name?’ And I say, ‘Oh, it’s like 617, but six-one fresh,” said Chris Marstall, a creative technologist in the Globe Lab. (Out-of-towners: 617 is the primary Boston area code.)

The name, like the site, is an experiment. Think of 61Fresh as a close cousin to something like Techmeme, Mediagazer, or our very own heat-seeking Twitter bot, Fuego. Where those algorithmic link aggregators are topic-based, 61Fresh instead has a geographic focus: “It’s philosophically the same,” Marstall says. “It’s using a network to find the most relevant stories in a narrow domain for now.”

Every 10 minutes, 61Fresh takes a spin around a localized slice of Twitter searching for rising news items. On any given day, you’re likely to find stories on apartment fires, startup companies, or the Bruins. Unlike some aggregators that mix in some human control, 61Fresh is 100 percent algorithm-driven — which means both that it runs without the need for ongoing staff resources and that the end product can be a little raw, with different versions of the same story repeated during breaking news.

The project is still in early stages, and Marstall said they’re still figuring out the best way for users to interact with 61Fresh, which for now exists as a website and Twitter feed. “The stories that pop up are stories drawing interest on Twitter,” he said. “It’s sort of an open question if this can be a standalone site in the future or if this is something that is a compelling part of”

But how does 61Fresh figure out what’s news, what’s talk, and what’s noise? The algorithm built by Globe Lab looks for tweets that link to stories or updates from just over 500 pre-selected websites in the Boston area. Aside from recognizable names like, and the Boston Herald, the universe also includes stories from radio, TV, town government sites, universities, and blogs, including the growing network of online outlets that cover the particular cross-section of education, technology, and culture unique to the Boston area.

“We got three daily newspapers, a bunch of weekly newspapers, we got TV and radio, and all these blogs. That’s just the tip of the iceburg,” Marstall said. “It shows there’s a lot of vibrancy here and a lot of smart people fighting to get attention for their stories.”

Here’s how they explain it:

We’re downloading a real time stream of the local Twitter posts (originating roughly within Route 128) that link to stories on one of those 500 sites. Every tweet represents a vote. Our algorithm mixes popularity with freshness and a couple other ingredients, discounting the value of ‘official’ accounts and bots. We group similar articles together, generating a diverse feed that changes frequently throughout the day.

What you get with 61Fresh is a cascade of headlines, but in the form 140 characters, as the links that are pulled in use the text of the most prominent tweet on a particular story. “We want this to look like a conversation, we don’t want this to look like a bunch of headlines,” Marstall said. Keeping in that theme, each story pulled into 61Fresh shows a roster of people who have tweeted the link, as well as Twitter prompts to reply, retweet, or favorite.

One interesting feature on 61Fresh is the “mute sports” button, which, as the name implies, lets users excise anything related to sports from their feed. “Early on, we realized the sports stories could really dominate,” Marstall said. “I like it — I’m a sports fan — but I know for a lot of people it’s total noise.”

Ali Hashmi, a former Knight fellow at the Globe Lab who is now at MIT, created a natural language processing system that helps the aggregator parse sports from the rest of the news. Using a base set of 1,000 sports stories, the system learns how to spot a sports article and pluck it out of the stream of stories, Marstall said, if you want neither pucks, punts, nor pitchers.

61Fresh is one of the latest projects to come out of the partnership between the Globe and MIT’s Center for Civic Media. Originally, the lab’s creative technologists and Knight fellows from MIT were working on a way to archive Globe reporters’ tweets from the Boston Marathon bombing. Lessons and code from that were put into use for 61Fresh, Marstall said.

This is not the first time the Globe Lab has experimented with social media feeds as a means of monitoring life in Boston. Previously the lab developed Snap, a tool that maps Instgram photos to their geolocated spots around the city.

Jeff Moriarty, the Globe’s vice president of digital products and general manager of, said 61Fresh “fits in with the idea that we’re moving to include more of what people are saying and what people are doing right now in Boston.” With the split between the two Globe sites, has been focused on news and information, but also digital tools and online discussion, Moriarty said. The central idea is to be able to give readers a sense of what they need to know in a format that works best for them. “By combining both the news and what’s popular on Twitter, you get this pretty unique view of what’s happening right now,” Moriarty said.

At the moment, 61Fresh is a little rough around the edges, the design is not particularly eye catching, and the team at Globe Lab is catching problems as they arise. But the lessons from the project will be applied in the future, either for a stand-alone aggregator or built into parts of Moriarty said he thinks the project can have a lot of value for niche topics like sports or technology — pulling in tweets and headlines for a Boston audience interested in the NFL or biotech, for instance. As Moriarty sees it, the job of isn’t just to deliver the news, but to provide context, conversation, and the other pieces to help readers see the bigger picture.

“I think it really needs to be what we do as a media organization: People want someone to help them make sense of what is going on around them,” Moriarty said.

POSTED     Sept. 23, 2013, 11:59 a.m.
Show comments  
Show tags
Join the 15,000 who get the freshest future-of-journalism news in our daily email.
The New York Times’ new Slack 2016 election bot sends readers’ questions straight to the newsroom
“Instead of asking you to come to us and be part of this massive room of people shouting over each other, you can bring us to you, and have us be, essentially, one more person in your conversation.”
The Conversation expands across the U.S., freshly funded by universities and foundations
The news site that uses academics as reporters and journalists as editors now boasts 19 paying member universities and is opening up posts in Atlanta (and maybe in the Bay Area).
A Boston public radio station is redesigning its site to make audio “a first-class citizen online”
But: “I’ve tried to be really disciplined about not calling this process just a redesign,” WBUR’s executive editor for digital Tiffany Campbell said. “We’ve built a brand new platform.”
What to read next
For McClatchy’s new head of news strategies, analytics and social traffic bolster good journalism
“It’s a beautiful thing when the powerful, public service, noble interests of journalism actually align with your business model.”
0The Information is offering members a perk: an exclusive trip to “meet the right people” in China
The $399-a-year site covering the tech industry expects subscribers to pay their own way, but promises access to “people only the most well-known execs typically meet.”
0From Nieman Reports: How news outlets are harnessing the power of video games for journalism
As digital technology allows more and more of our lives as consumers to be framed as play, scoring points or competing with others, companies of all kinds have been incorporating games into their strategies.
These stories are our most popular on Twitter over the past 30 days.
See all our most recent pieces ➚
Fuego is our heat-seeking Twitter bot, tracking the links the future-of-journalism crowd is talking about most on Twitter.
Here are a few of the top links Fuego’s currently watching.   Get the full Fuego ➚
Encyclo is our encyclopedia of the future of news, chronicling the key players in journalism’s evolution.
Here are a few of the entries you’ll find in Encyclo.   Get the full Encyclo ➚
The New Yorker
Daily Mail
Investigative News Network