Nieman Foundation at Harvard
HOME
          
LATEST STORY
Deepfake detection improves when using algorithms that are more aware of demographic diversity
ABOUT                    SUBSCRIBE
March 2, 2017, 1:52 p.m.

For Quartz, bots are a chance to build a new path for interacting with news (and news outlets)

“It’s basically like with anything that you do in media: Is it interesting? Is it easily accessible? Is it compelling? Are there reasons for me to come back?”

John Keefe recently joined Quartz as a developer for its new Bot Studio and as the project manager for its app.

He’s been there for three weeks now — though he is quick to point out that he was out of the office for one of them so it’s really only been two weeks — but that’s not stopping Quartz from launching one of the Bot Studio’s first experiments, a Twitter bot to help attendees of the NICAR conference.

Keefe says the Bot Studio, which is funded by a $240,000 grant from the Knight Foundation, will focus on two main areas: How bots and artificial intelligence can help journalists do their jobs better and how news consumers can use them to access news and information. (Knight also supports Nieman Lab.)

Keefe joined Quartz after spending 16 years at WNYC. We spoke about the lessons he learned from public radio, his goals for the Bot Studio, and what it’ll take for bots to catch on. Here’s an edited and condensed version of our conversation.

Joseph Lichterman: Can you tell me about the bot you’re unveiling at NICAR?

John Keefe: “Unveiling” may be a little too dramatic of a description of it. Basically, in the last couple of weeks, we’ve created a conference bot, a Twitter bot — @newsnerdbot. We scraped the conference website for all the schedule information and speakers and then fed that into a database, and then wired that up to a natural-language processor, and then wired that up to Twitter. Basically, the idea is that we can try to discern what kind of questions you’re asking about the conference, and then it would reply back. It’s absolutely an experiment — it’s really rough. We tried to make it friendly, but it has a lot to learn. If you ask a question of it, it will try to discern what kind of question you’re asking, figure out the key components of that question, and then reply. It’s up and running right now. We’re not so much as unveiling it as to making it available for people to give it a shot. We hope that people will give it a try.

Lichterman: So this is really your first experiment with the bot studio. How does it fit into with what you’re trying to achieve there?

Keefe: The whole idea of the bot studio is making useful bots for journalists and their audiences. This is kind of a first stab at it. It just happens to be a conference of journalists that we’re experimenting with.

One of the reasons everyone is talking about bots is because they’re so much easier to make these days, and the kind of things we expect from a bot — its ability to understand natural language when we ask it a question — that’s becoming more accessible to people who are not experts in artificial intelligence. There are services out there which you can train to start to understand the types of questions your bot may face. Part of it is experimenting with some of those services.

The one that this bot uses is called API.ai. We were playing around with a couple of the other ones. Part of it is to get a sense of what base tools might be most useful, how we would go about using them, and how quickly we could whip something together. I’m very much a proponent of experimentation and prototyping. For the last several years, I’ve been experimenting with all kinds of things. I like to try new things and try them quickly to get a sense before diving too deeply into one path. That’s really what this is. It’s an experiment to test out a couple of the natural language processing services and figure out how we would wire them up. They’re all a little bit different and they’re all pretty new, and they all use words like intents and entities and different terms to describes how their models are working. It’s all new, and it’s all relatively new to journalism, and newish to me and lots of folks. It’s all going to be really important to making things for audiences and journalists.

We’re looking into making useful Slack bots for journalists and looking at what we might do with voice command applications like the Google Homes, Alexas, and Siris of the worlds. All of that is going to require some of this understanding of natural language, and so that’s what this is really about testing out.

One of the ways we define bots is “software you can talk to.” Behind the scenes, you need to have good software that will understand what you’re saying. That’s what we’re experimenting with.

Lichterman: The Bot Studio is still pretty new — I’m curious where you’re thinking of going from here.

Keefe: You’re absolutely right. It’s brand new. I’ve only been here now three weeks, and for one of those weeks, I was out of the office. It was certainly not the first bot Quartz has made — internally or externally facing.

Our hope is to build things that are useful — experiment with this notion of not only for journalists to do their job better, but then how do the news organizations’ audiences interact with the content? How do we do that? What is the best way to do that? We are so early in that era in news in general. Being able to ask Alexa for your latest news, which you can now do, or to chat with a bot in a text-message kind of way.

Quartz really led with this with its app, but the app, when you use it, you just have a couple of options each time at the point of interaction. What if that system had more open-ended, freeform text input? What would that look like? What would people want to know? How would people ask questions? Do people even want a freeform text bot? That is something that needs to be tested out. This is all stuff that we’re going to be playing with.

We have some early ideas about ways we might do that, both for tools for journalists and for ways for interacting with the public. Our plan is actually to roll out these prototypes — basically posting them on GitHub and posting them on our blog talking about them, talking about what we’re learning and how we’re building them, opening the code up for people to see and also for people who want to experiment with us. Not necessarily in a “here’s how to install it” kind of way, but with prototyping and experimentation. If you can understand the code or you want to interact with us, take a look and see what we’re trying to do. The ultimate goal is to actually have useful, rock-solid bots that will be available to journalists and some that will be audience-facing. We’re going to experiment with how to interact in this way through bots with Quartz’s content for sure and then share that out to say here’s how we’re doing that and then to make it available for anyone to use. That’s all part of the Knight grant that Quartz got, which is part of bringing me here on board.

Lichterman: You mentioned that you have some early ideas about how you might achieve those goals. What types of things are you planning?

Keefe: Like I said, I’m basically two weeks in. At the moment, they’re unformed conversations. It’s pretty clear that journalists in the U.S. and elsewhere are using Slack as a tool to communicate with each other and to get information. Can we figure out ways to bring other information to reporters’ finger tips using Slack?

A bot, of course, is how you talk in Slack. You talk to each other and you talk to bots. There are plenty of bots that are designed for Slack. It’s a fairly easy platform to make bots for, and since journalists are really into using Slack, that seems to make sense. Are there things we can help journalists do in Slack, whether it’s covering their beat better or staying abreast of information? It’s not entirely clear where that will end up, but those will be part of the early experiments. I have been to talking to journalists here at Quartz just to get a sense of the things they want to keep track of, the types of things they might want to do, so some sort of tool in Slack that journalists could use is one way to do their work better. That’s one way for sure. Exactly what that is, we’re still kicking it around. But the kicking-that-around will happen in public, which is kind of cool.

Another area is in the voice interaction world. What does it mean to speak to a bot for an audience person? This is for someone who is consuming Quartz content. How might you do that verbally? What makes sense? What’s a Quartzy way to communicate with us and our bots to get access to our information? Alexa has the Alexa skills and the flash briefings — you can ask Alexa what the news is today, and Quartz can be one of the briefings that you add in so you can hear our content. Right now, what we’re doing is we’re giving Alexa information to read to you, so you hear Alexa’s voice. Is that the best way to get your Quartz information, your Quartz news? Maybe, maybe not. So we’re going to experiment with that as well.

Then the question becomes how you interact with Alexa, Google Home, or any of these voice-response services to get information. Can we be creative about that? Can we be super useful about that? This whole industry is new. This is all sort of emerging. We’re definitely going to play with this voice interaction to get access to our content. Other folks are doing that already. We’re going to be doing it and putting it out there in public for people to play with it.

The third area is really where my two roles here as the bot developer and the product manager for the app really overlap — this free-text interaction. What works with that? There are lots of messaging platforms now. We’re looking at those. We obviously have our own app that you sort of message with, but in a really limited way. Could we open that up? What would people want? What would make sense? How do we keep it so it’s in the spirit of the app, which is a mix of news and sparkly information for your day and animated GIFs and emojis? How do we keep that spirit if we switch to the notion that you could communicate back and text or send an emoji back? What do we want to do? The textual interaction with our content. How best to do that? I think a lot of places are wrestling with it. But those are the main areas that we’re looking at, and we’ll be sharing out what we learned in the process.

Lichterman: Attention to bots reached a new level in 2016. Facebook added bots to Messenger, for instance, but a lot of news orgs’ efforts there have been disappointing. What do you think it’ll take for bots to really become mainstream?

Keefe: First of all, I think this needs to be determined. What I say is speculation at this point. But my sense is that it has to be things that are going to get your attention, hold your attention, and be a great experience.

So that’s kind of a high bar, right? It has to be a really good experience. It has to be a positive experience. And in the news world, also a useful experience. There’s a novelty to things that is important to get your attention, but how do people stay? I think people stay because things are useful. They have a good spirit to them. If it becomes a necessity, that’s really great.

We have an Amazon Echo at home, and watching how my family plays with Alexa is really interesting to me. It’s completely useful for playing songs. It’s fun for cat facts — my daughter likes cats, so every day we hear a new cat fact, and that’s useful and playful for her. How do we maintain a sense of usefulness and playfulness that brings people back to use these interfaces? That’s really key.

Also, as people interacting with machines, we’re going to expect a high level of understanding, and I think frustration when our devices don’t understand what we mean puts a lot of burden on those of us trying to make those devices or code those devices. That’s going to be a challenge. It’s basically like with anything that you do in media: Is it interesting? Is it easily accessible? Is it compelling? Are there reasons for me to come back? You could have asked that same question about the web. Now that the web is becoming more popular, how will news end up on websites? How will you get people to interact with it? It’s all going to be the same thing. You need to be compelling, useful, accurate, and if you up the quality — that’s what we’re banking on — that’s going to be key to set you apart.

The one thing I would say, and I would have said this even before coming to Quartz, is that Quartz is amazing at user interaction. How Quartz and you connect is super important. I remember Zach Seward talking about this early on, even before Quartz was launched, that this was going to be key. It’s really part of the entire operation here: How do people interact with Quartz?

That’s what’s so cool about the app, the morning email, and the website. The app made this big splash partly because it’s not what you’d expect from a news site. It’s a chatty, text-like app. It’s not “here’s a list of headlines, click on one.” I really liked how Quartz has already looked at user interaction, tried to make it as accessible as possible, and that’s what we’re going to try and bring to these conversational interfaces. In that way, whatever lessons we’re able to tease out here at Quartz are probably going to be pretty useful because there’s such an attention to user interaction here. That’s a long round about way of trying to answer your question, but the same qualities that you might see in your favorite websites or your favorite podcasts, are going to be what make the day in the world of conversational interfaces.

Lichterman: You were at WNYC for many years. Were there any lessons or takeaways that stick out to you as you’ve moved into this new role?

Keefe: Oh, for sure. I had 16 years of adventures and learning at WNYC, so I’m bringing all of that with me here. Particularly relevant to the area of conversational interfaces and software you can talk to is voice, audio, and the act of listening and responding. One of the things I always love about the best public radio and the best podcasts is that when you’re listening to an episode of say, Radiolab, that’s a one-way thing. You’re listening to the hosts Jad and Robert talk about science and other things. In the process, they often get you to participate. You actually do participate when you listen to that great audio content. In your mind, you’re running through that ethical dilemma or you’re picturing the landscape, whatever you’re doing, you’re participating in that experience.

The question then becomes, in this aural, explanatory way that I’m already engaged in through my mind, what if I can engage in that with my voice? That’s going to be something that’s incredibly interesting. Maybe the answer is: It doesn’t work! That could be an outcome. But the beauty and skill that my colleagues at WNYC have used to engage the listener without the listener being able to engage back is pretty amazing. Since so much of this is audio-based and about listening then talking back, that whole listening part is something I carry with me from what I saw and learned at WNYC.

That, plus the spirit of innovation. The opportunity I was given for many years to break new ground and experiment with things online, with maps and charts and things — things we now all see on lots of websites, when five or six years ago that was a little bit more novel. The opportunity to play in the new and novel when it comes to information, presentation, and journalism is absolutely what I’m bringing to Quartz and looking to experiment with new interfaces and new possibilities when it comes to information and journalism.

Lichterman: Is there anything else I should’ve asked about or something I missed?

Keefe: One of the big emerging themes coming out of this world of bots is not just the botness but the artificial intelligence. AI is being used, and has been used for a long time, to serve ads and to do other things, by Google, Facebook, and the big players. The notion that artificial intelligence in this field could actually apply to journalism and journalists in some way [is exciting]. It may take the form of bots and natural language processing, but there are so many other areas of artificial intelligence and machine learning that may apply to journalism, that this is just the tip of the iceberg.

Top photo by Michael Dain used under a Creative Commons license. Photo of John Keefe manspreading on the New York City subway — there’s an explanation — by WNYC also used under a Creative Commons license.

POSTED     March 2, 2017, 1:52 p.m.
Show tags
 
Join the 60,000 who get the freshest future-of-journalism news in our daily email.
Deepfake detection improves when using algorithms that are more aware of demographic diversity
“Our research addresses deepfake detection algorithms’ fairness, rather than just attempting to balance the data. It offers a new approach to algorithm design that considers demographic fairness as a core aspect.”
What it takes to run a metro newspaper in the digital era, according to four top editors
“People will pay you to make their lives easier, even when it comes to telling them which burrito to eat.”
Newsweek is making generative AI a fixture in its newsroom
The legacy publication is leaning on AI for video production, a new breaking news team, and first drafts of some stories.