Nieman Foundation at Harvard
HOME
          
LATEST STORY
From shrimp Jesus to fake self-portraits, AI-generated images have become the latest form of social media spam
ABOUT                    SUBSCRIBE
Feb. 29, 2016, 10:21 a.m.
Mobile & Apps

WNYC’s Note to Self sent 300,000 texts to 15,000 people; here’s how they did it and what they learned

It was for an experiment measuring information overload: “There’s a conversation going on between me and my listeners, but I want to make it as frictionless as possible for them to get their thoughts and worries to us.”

Receiving 1,700 voicemails might sound like a nightmare to some [raises hand]. For Manoush Zomorodi, the host of WNYC podcast show and podcast Note to Self, it was an “incredibly gratifying” sign of a job well done.

In February, Note to Self launched Infomagical, a week-long series designed to help listeners free themselves from information overload — you know, the inevitable side effect of a life spent glued to your phone.

Infomagical Note to Self WNYC square art John HerseyNote to Self had run a similar project, Bored and Brilliant, in 2015, in which the show partnered with two different apps to let listeners track how much time they spent on their phones. This time around, Zomorodi and John Keefe, WNYC’s head of data news, were interested in interacting with readers via direct texting. Zomorodi had been reading about conversational commerce — the idea that businesses can move beyond apps by texting directly with consumers. “We’re a public radio station, so it’s a little different,” she said. “But we loved the idea of not only being able to talk directly to our listeners, but that they would be able to talk back to us by rating themselves, contributing data, and leaving voicemails.”

Keefe thought back to a 2014 WNYC project, Clock Your Sleep, where the station asked users to keep track of how much and how well they slept. For that project, WNYC built a standalone app. While Keefe didn’t want to build an app for Infomagical — “too arduous” — he did love the app’s ability to push notifications out to listeners.

“The whole trick with the sleep project was to get people to record their previous night’s sleep every day. If you don’t have a reminder, you’re going to blow it off,” Keefe explained. “You could set the app to give you a reminder as you were going to bed, or as you woke up. That kind of persistent reminder turned out to be incredibly useful for people who want to participate in a project like this.” The sleep data that WNYC collected through its app with those push notification reminders turned out to be much more reliable than data collected via the show’s website or via fitness trackers.

So for Infomagical, the team decided to create a texting system, because text reminders can fill the role of push notifications from an app. Keefe’s team, lead by programmer Alan Palazzolo, used a service called Twilio that can program and send texts automatically. To begin, listeners were asked to set information overload goals for themselves and provide their phone numbers on the Note to Self website. They then received followup texts at the phone numbers they’d entered, with some questions about their goals (those answers were used to sort data later). The texts were adjusted for time zone, since “there’s nothing like being asked to join a project at four in the morning.”

Next, the data team worked with Zomorodi to figure out how many text messages (or emails) participants should receive. They ran a test on a few WNYC employees to test timing; while they’d originally planned to send the first text of the day in the late morning, testers said that timing was bad because they were already involved in their days. Instead, the team decided to send the texts earlier in the morning, before they began their work days.

Infomagical_BotEach morning, listeners received a podcast link. (New Infomagical episodes ran daily from Monday, February 1 to Friday, February 5.) On three days out of the five, they received a midday text asking how things were going. “We figured out how to text back encouraging GIFs to keep you encouraged,” Keefe said. At the end of the day, participants were texted two questions: Whether their level of information overload felt more, less, or the same, and how much progress they were making with their goal (on a scale of 1 to 5, with 5 being great). On two days out of five, participants were invited to leave a voice message for Note to Self. Those responses were then included in the final episode of the series.

“We were worried about annoying people with texts, but in retrospect, I wish we’d sent them a reminder every single day,” Zomorodi said. “People would say they’d forgotten and we got them back on track. Notifications are annoying if you don’t ask for them, but if this is a week when you decide you want to change something in your life, you want as much help as you can get.”

One way the team avoided irritating people was by making sure that the texts were written in Zomorodi’s voice, the voice of the show, “so it really did feel it was Note to Self reaching out to you, not a company that wants something from you,” Zomorodi explained. Ariana Tobin, the show’s producer, wrote the text for the podcast, newsletter, web copy, and text messages, making the 160 characters available in a text “really feel crafted and kind and gentle, while actually bringing across some information and extracting some information at the same time.”

Radio and texting actually have a lot of parallels, Keefe pointed out. “Who do you text with? You text with your family and your friends, and you don’t get too many bot texts,” he said. (At least for now.) “You’re usually communicating one-on-one with somebody. The funny thing is that people also perceive radio and podcasting as being a one-on-one conversation: ‘Here’s Soterios Johnson doing Morning Edition in my kitchen.’ You don’t think of that as a broadcast, but a companion that, for years, has been one-on-one. That is sort of mirrored in the texting. Yeah, we’re sending this to everyone who’s participating, but the experience is much more of a one-on-one interaction, and when you answer the question, it responds right back to you with some encouragement.”

The one-on-one, personal communication concept was echoed in the voicemail prompts, which invited listeners to say more about the goals they’d chosen for themselves. When they texted back that they were interested, their phones would ring and they’d pick up to hear a recording of Zomorodi thanking them for participating. “There are other shows that set up voice lines, but this was different,” Zamorodi said. “You were literally being called by us and talking back.”

Listeners ended up leaving 1,700 voicemails in that first week. Zomorodi was able to listen to most of them, she said, and excerpts from several hundred of the voicemails were included in the final “results” episode.

“When people leave comments on the website, there’s an expectation that the public is going to see them,” Zomorodi said. “You’re presenting your ideas for public consumption.” The voicemails were, by contrast, if not confessional, much more intimate. “It was like a friend leaving me messages,” she said. “A lot of them would start out by being like, ‘Hey, Manoush, here’s what’s going on in my day.’ At the end, some people would say, ‘Okay, I’ll talk to you again tomorrow.’ I was really moved by it in a way that I had not expected to be.”

On the coding and developer side, the project was “a big lift,” Keefe said. “When we’re talking about 15,000 people we’re talking to during the week, that’s like 300,000, 400,000 texts,” he said. When a company sends that many texts, wireless providers may spam-block its texts. To get around this, WNYC could have applied for a short code “so that the phone companies and all the carriers know you’re not a spammer.” But it takes time to go through the application process for a short code and the codes themselves are expesnive, around $3,000 a quarter, Keefe said.

The texts also had to be queued up. And it costs around half a cent to send a text, which adds up when you’re texting thousands of people three times a day. Ultimately, keeping all of these considerations in mind and in order not to overload its servers, WNYC decided on the 15,000-person cap for the first week.

Since that week ended up running smoothly, and since so much work had gone into it, WNYC decided to run the Infomagical campaign again. In fact, it will run every week, starting again each Monday, “until we feel like we don’t need it anymore,” Zomorodi said. “We’re curious to see if this becomes an underground moment: Like, you’ve got to do an Infomagical week!” When we spoke, several thousand people had already signed up for the new week. And, yes, Zomorodi promised she’ll be listening to all the new voicemails she gets.

“There’s a conversation going on between me and my listeners, but I want to make it as frictionless as possible for them to get their thoughts and worries to us,” she said. “This seems like a great way to do it.”

Art by John Hersey, emojis by Kevin McCauley. Used with permission.

Laura Hazard Owen is the editor of Nieman Lab. You can reach her via email (laura_owen@harvard.edu) or Twitter DM (@laurahazardowen).
POSTED     Feb. 29, 2016, 10:21 a.m.
SEE MORE ON Mobile & Apps
Show tags
 
Join the 60,000 who get the freshest future-of-journalism news in our daily email.
From shrimp Jesus to fake self-portraits, AI-generated images have become the latest form of social media spam
Within days of visiting the pages — and without commenting on, liking, or following any of the material — Facebook’s algorithm recommended reams of other AI-generated content.
What journalists and independent creators can learn from each other
“The question is not about the topics but how you approach the topics.”
Deepfake detection improves when using algorithms that are more aware of demographic diversity
“Our research addresses deepfake detection algorithms’ fairness, rather than just attempting to balance the data. It offers a new approach to algorithm design that considers demographic fairness as a core aspect.”