We’ve been testing [emerging story alerts] over the past several months. A good example [of an emerging story] was the Amtrak train derailment in Philadelphia. We saw a lot of early reports that were unsubstantiated, but lent more credence to the possibility that this was a big deal.

Amtrak train partially derails in Philadelphia; injuries reported – @Jim_Gardner, @PatrickMurphyPA http://t.co/jVJJBI3iSn

— Breaking News (@BreakingNews) May 13, 2015

If you have those topics that corresponded to it selected, you got a notification. Those who were in the area of the derailment, in Philadelphia, would have gotten a proximity alert as well. Several minutes after the proximity alert, we sent a test of the emerging story alert. From there, it was several minutes after that, [the story] had enough momentum, we had confirmed information that it was a big deal, and it was a global push.

Not every story follows that path. A lot of emerging stories don’t turn into a big enough deal to prompt a global push. If a story is trending, it’s too late for us — we’ve missed our opportunity.

We’re unabashedly hard news. We do not care about driving open rate. We care about getting important news to people who need it and want it. Our alerts are very straightforward; we tend to pack as much information into them as we possibly can. It’s a win for us if you can just glance at your phone or your watch and get the information you need. We just want to tell you and save you time and send you on your way.

Every story is a judgment call, a risk assessment. In the case of the Supreme Court [decision on same-sex marriage], we knew that there was going to be a ruling. However, we’ve had some problems in the past with news organizations getting rulings wrong, so typically, on big Supreme Court rulings that will result in a global push, we wait for three solid sources to agree before going with it.

Among the sources we look at, we know, historically, which ones tend to be more accurate than others, and those get a little more weight in our decision-making.

We discount some information. We apply editorial judgment. Picture yourself as a reporter in the early stages of a story. Eyewitnesses tell you what happened, and then a cop comes up and tells you one little piece of information. You have to make a judgment call on what you’re going to report. This is very similar to that, but we’re doing it all over communication channels instead of physically being there.

We’ve always believed that the best products can be built by combining algorithms with people in seamless ways. When we enter an item our CMS, a variety of [automated] services make an educated guess on which topics and stories this thing that was just typed in applies to. Then our editors can very quickly edit the estimation by the CMS. It can be sometimes a little tedious, but it’s a very fast platform set up for speed. It’s proven to be very effective for us.

There’s a lot of nuance in topics, with locations especially. That’s why we’ve spent so much time and effort building the CMS and creating a consistent style guide. Our style guide is 80 percent about topics and location. Once there’s accuracy and consistency there, that drives all those push alerts. There’s also lots of metadata that goes beyond topic, but I’m not going to get into that — that’s the secret sauce — but different stories have different weights, and that all comes into play.