Nieman Foundation at Harvard
PressPad, an attempt to bring some class diversity to posh British journalism, is shutting down
ABOUT                    SUBSCRIBE
Oct. 14, 2015, 9:47 a.m.
Reporting & Production

Faster, stronger, and with better kickers: The New York Times’ 4th Down Bot gets an upgrade

The bot, which analyzes play calling by NFL coaches in pressure situations, now factors in team strength and the increasing accuracy of kickers. It’s also now open source.

The Vikings were losing, again. It was late in the fourth quarter and Minnesota was down 10 against the Denver Broncos and a cardboard cutout of Peyton Manning. This is a familiar scenario to me, one I have trouble dealing with objectively. So when the Vikes decided to go for it on 4th and 1, I had no small amount of terror climbing up my back.

Clearly, I should have been listening to The New York Times’ 4th Down Bot, who knew the Vikings had a 67 percent chance of getting the first down. And after they converted, they went on to score.

They still lost the game. Some things bots still can’t fix.

The Times has let 4th Down Bot call the shots on NFL games since the 2013 season. Using a combination of historical data, real-time stats, and just a little attitude, the bot attempts to make the most informed decision possible on what your favorite team should do when its coach has to make a tough call.

But like any third-year NFL veteran, 4th Down Bot has gotten a little smarter about its game. This fall, the team behind the bot has made some upgrades to the way the system reads games and makes calls on the field. The bot is now set up to make faster calls, take better account for the power of underdogs, and do more to predict the behavior of one of the most important players on the field: the kicker.

“The older model was built taking advantage of the past 15 years of data, but it wasn’t accounting for a lot of the improvements that kickers have made,” said Josh Katz, a graphics editor on the Times Upshot team who helped reboot the bot.

Yes, that’s right, kickers, seemingly the least important player on the bench. The position you likely pick next to last on your fantasy team. It’s become one of the most important jobs on the field as kickers have become increasingly accurate. Ben Morris, writing in FiveThirtyEight, sums it up:

For all the talk of West Coast offenses, the invention of the pro formation, the wildcat, 5-wide sets, the rise of the pass-catching tight-end, Bill Walsh, the Greatest Show On Turf, and the general recognition that passing, passing and more passing is the best way to score in football, half the improvement in scoring in the past 50-plus years of NFL history has come solely from field-goal kickers kicking more accurately.

Now the bot takes into account the improved accuracy and distance of kickers, but also additional variables like the stadium (indoor vs. outdoor) and weather forecast (clear skies or blustery).

The 4th Down Bot now also accounts for the relative strength of a team against given opponents. Specifically the bot looks out for underdogs, giving a one-touchdown underdog a 22 percent chance of winning at the start of the game. That makes a difference to the bot because underdogs may be more likely to be aggressive on fourth-down plays, said Trey Causey, a data scientist who worked with the Times on upgrading the 4th Down Bot.

The secret? Gamblers. The bot’s new team strength predictor is based on the point spread of individual games, Causey told me. “You can’t do much better than hundreds or thousands of people who have made their thoughts about who they think is going to win available,” he said.

The trickiest part of the updates to the 4th Down Bot was speeding up its play-calling ability. This fall they introduced “live calls,” where the bot makes a suggestion before a team makes its fourth-down play. As Causey explains, the bot runs through the likelihood of success or failure on a punt, field goal, or going for it and makes a call right after the third down is complete.

But the live calls also introduce an element of risk for the bot. There are some variables the 4th Down Bot can’t account for, like penalties or coaches’ challenges that change a team’s field position. The live calls are automated, meaning the bot could base its call off incomplete information before the next snap of the ball.

Causey and Katz say they’re still testing out the live calls feature and can turn it on or off during the season. “I think it’s fun,” Causey said. “I also think it gives good insight into how fast these decisions have to be made and how much of the process is done on the fly.”

The upgrade to the bot came about partially because they wanted to refresh the project, but also because Brian Burke, who created the original model the bot uses, went to work for ESPN.

“Brian going to ESPN forced us to change some things about how we were running the bot,” Katz said. “But at the same time, there were a few aspects of the bot we wanted to change.”

Those changes also presented a new opportunity to give other people ways to improve the 4th Down Bot by making the system open. Well, open with a small caveat: The code responsible for the bot’s prediction model is now available on GitHub, but it will only really work with a subscription to NFL data from Armchair Analysis.

Katz said the Upshot team likes to make code open if there’s no reason, legal or otherwise, not to make it available online. (One example of the benefits of being open: Developer Ben Dilday has already forked the bot’s code to create a version that works with freely available NFL play-by-play data.)

In the case of the bot, putting the code on GitHub is a way of being transparent about how the system works to all the people who have come to use it.

“It turns something from what would be a black-box mystery to: Here’s exactly where these numbers come from,” Katz said. “It’s not this mysterious, mystical, process. It’s just math.”

Photo of an electric football game by Steven Snodgrass used under a Creative Commons license.

POSTED     Oct. 14, 2015, 9:47 a.m.
SEE MORE ON Reporting & Production
Show tags
Join the 60,000 who get the freshest future-of-journalism news in our daily email.
PressPad, an attempt to bring some class diversity to posh British journalism, is shutting down
“While there is even more need for this intervention than when we began the project, the initiative needs more resources than the current team can provide.”
Is the Texas Tribune an example or an exception? A conversation with Evan Smith about earned income
“I think risk aversion is the thing that’s killing our business right now.”
The California Journalism Preservation Act would do more harm than good. Here’s how the state might better help news
“If there are resources to be put to work, we must ask where those resources should come from, who should receive them, and on what basis they should be distributed.”