HOME
          
LATEST STORY
Why The Daily Pennsylvanian is spending $100,000 over the next two years to foster innovation
ABOUT                    SUBSCRIBE
April 1, 2013, 10:18 a.m.
Audience & Social

Not an April Fool’s joke: The New York Times has built a haiku bot

Times Haiku are generated from stories on the homepage of NYTimes.com, just in time for National Poetry Month.

timeshaiku2

New York Times senior software architect Jacob Harris has a thing for robots and wordplay. You may recall he’s the guy behind @nytimes_ebooks, the Times answer to the elusive and inscrutable Twitter bot @Horse_ebooks.

So it’s only natural that Harris has now created an algorithm that extrudes haiku out of the text of Times stories. In other words:

Haiku harvester
built inside The New York Times —
does it have a soul?

(If my eighth grade English teacher is reading this. Sorry.)

Here’s a better, more Times-y example:

timeshaiku1

Times Haiku is a collection of what they are calling “serendipitous poetry,” derived from stories that have made the homepage of NYTimes.com. The haiku live on a Tumblr hosted by the Times. Harris built a script that mines stories for haiku-friendly words and then reassembles them into poetry. (For those of you that may have zoned out in class, haiku are comprised of three lines with, in order, five, seven, and five syllables.) The code checks words against an open source pronunciation dictionary, which handily also contains syllable counts.

“Sometimes it can be an ordinary sentence in context, but pulled out of context it has a strange comedy or beauty to it,” Harris said.

Harris was inspired by Haikuleaks, a similar project that found poetry in the cache of diplomatic cables released by WikiLeaks in 2010. The backbone of that project was an open source program called Haiku Finder, which crawls through text to generate haiku. The program was built in Python; Harris made his own version in Ruby on Rails.

The result, much like @nytimes_ebooks, is bizarre, quirky, and kind of zen. The haiku have a strange way of getting at the heart of a story, or teasing out interesting fragments from an article. “There’s something appealing about finding these snippets of text, these turns of phrase and pulling them out,” Harris said. “You find it compelling and it drives you to read the article that it came from.” (Think of it as a more lexicographically strict version of Paul Ford’s SavePublishing.)

In its own poetic way, Times Haiku will be another access point for Times stories, said Marc Lavallee, assistant editor for interactive news at the Times. “If someone sees the site, or the image of an individual haiku and shares it on Tumblr, and it gets them to think about who we are and what we do, or gives them a moment of pause, I think we’ve succeeded in a way,” Lavallee said.

Lexi Mainland, social media editor for the Times, said they wanted the poems to be able to stand on their own and be readily sharable. That’s why the haiku are actually images, which fits well with the aesthetic of Tumblr, she said. Outside of Tumblr, the Times will promote the haiku through the paper’s flagship Twitter account.

That the Times has the ability to build a haiku bot isn’t surprising. But why build a haiku bot? “A lot of the projects we work on here are these incredibly big heaves, which are very, very gratifying,” said Mainland. “But you crave these smaller projects, which are just as valuable.” Similarly, projects like the haiku bot may seem silly on the surface, but the underlying code, the use of natural language processing, or other components could be valuable to future projects, Lavallee said.

It helps that the project came at little expense to the Times — Harris put it together on his own during a fit of post-election letdown. Harris had been working on projects connected to the presidential race for over a year, and after election day suddenly found himself with idle hands. He wrote the code in November and began monitoring what it was spitting out. After showing it to Mainland, Lavallee, and other editors, they gave the project a green light. Designer Heena Ko and software developer Anjali Bhojani gave the haiku their distinctive appearance for Tumblr. (Those lines you see running askew of the text of the haiku? The length is computer generated, based on the meter of the first line of text.)

As whimsical as a haiku bot or a spammy-sounding Twitter bot might be, both are efforts to find new uses for the Times’ vast collection of work. “It’s just this large corpus of text that gets very dizzing to look through,” Harris said.

The Times may also have a soft spot for artwork inspired by the written word. Anyone who has visited the lobby of The New York Times Building has likely seen Moveable Type, an algorithm-backed art installation that displays fragments of Times content across 560 display screens.

But why poetry? For starters, today is the first day of National Poetry Month, Mainland said. (Today is also April Fool’s — and if you were wondering, this is not a joke.) Still, for lovers of verse, it may sound like a cold and bloodless way to create poetry. Can you really create poetry without a soul? Do robots have feelings? Can they really see a sunset, or be moved by the sounds of a whale songs CD?

Harris admits the bot is imperfect; it’s required a little teaching along the way. One reason he limited the scope to the front page was because it provides an editor-picked selection, which tends to be richer features and important daily fare. (Running the bot on the Times Wire, Harris said he often got haiku made up of basketball scores, which may be too esoteric for any lit major or stat nerd.) The algorithm is designed to toss haiku with certain sentence constructions (sentences that start with a preposition, for instance) or from sensitive stories. Mainland, Lavallee, and Harris also keep an eye on the haiku being created to see if anything untoward sneaks through.

But Harris also has to do some syllable counting himself, teaching the bot words that appear in the Times (“Rihanna,” for instance) that it doesn’t know. Henry Higgins would be proud.

POSTED     April 1, 2013, 10:18 a.m.
SEE MORE ON Audience & Social
SHARE THIS STORY
   
Show comments  
Show tags
 
Join the 15,000 who get the freshest future-of-journalism news in our daily email.
Why The Daily Pennsylvanian is spending $100,000 over the next two years to foster innovation
The University of Pennsylvania student newspaper is looking for innovative students on its staff — and from outside the paper.
Q&A: The FT’s Gillian Tett on separating digital from print and tailoring news to new reading habits
“What is changing is people are actually saying, Okay, how are consumers, our readers, actually consuming the news?”
The newsonomics of auctioning off Digital First’s newspapers (and California schemin’)
More than 200 newspapers are up for sale — as one group, in clusters, or one by one. Where they go could have a big impact on how the industry will look in the coming years.
What to read next
751
tweets
Wearables could make the “glance” a new subatomic unit of news
“The audience wants to go faster. This can’t be solved with responsive design; it demands an original approach, certainly at the start.”
677Designer or journalist: Who shapes the news you read in your favorite apps?
A new study looks at how engineers and designers from companies like Storify, Zite, and Google News see their work as similar — and different — from traditional journalism.
596Ken Doctor: Guardian Space & Guardian Membership, playing the physical/digital continuum
The Guardian is making its biggest bet on memberships and events by renovating a 30,000 square foot space to host live activities in the heart of London.
These stories are our most popular on Twitter over the past 30 days.
See all our most recent pieces ➚
Encyclo is our encyclopedia of the future of news, chronicling the key players in journalism’s evolution.
Here are a few of the entries you’ll find in Encyclo.   Get the full Encyclo ➚
Crosscut
Chi-Town Daily News
AOL
MinnPost
New West
St. Louis Beacon
PBS
New England Center for Investigative Reporting
American Independent News Network
CBS News
Reddit
The Nation