Can publishers foster quality conversations? About politics? What formats encourage respect and genuine dialogue? Can group chats make a reader more willing to support a publication financially?

The Center for Media Engagement partnered with Vox and Spaceship Media to explore these questions. They invited members of The Weeds Facebook group — a group for listeners of the Vox policy and politics podcast hosted by new Substacker Matthew Yglesias — to receive a one-hour training and join small group chats on Facebook messenger.

Of more than 20,000 group members, 184 volunteered and half were invited to participate. (Ultimately, 61 members actually attended a training session.) The participants were encouraged to “deeply listen to one another” and to “express kindness, curiosity, and empathy in their conversations” before being divided into six small group chats.

Every week for six weeks, small group chat participants received an email asking them to listen to a new episode of The Weeds podcast and discuss the episode in their group chat. Spaceship Media crafted discussion prompts for each week that touched on the major themes from the episodes, which included the Democratic primary, racial inequality, and how coronavirus could impact healthcare and the economy. Starting on March 25, the moderator sent the prompts to each group chat every Wednesday afternoon. On the same day, the moderator also posted the prompts in the larger Facebook group.

So how did it go?

Members of the group chats reported that they were less likely to self-censor — withhold their true opinions — in their small group chats than in the Facebook group. Overall, participants felt a stronger connection to members of their chats than members of The Weeds Facebook group and rated their fellow participants as more respectful and civil than commenters in the larger group. In the group chat, members said they were more likely to speak their mind but felt that — even in the smaller format — conversations were dominated by a few people just as often.

People in the group chats asked more “genuine questions” than commenters in the large Facebook group. There were more “sarcastic, inflammatory, and leading questions” in the comments of the large Facebook group.

People in the group chats tended to use different types of evidence to support a position than those in the large Facebook group. Those in the group chats were more likely to mention books, podcasts, or movies — and their own personal experience. (“Examples of personal information included their interactions with the healthcare system, their struggles adjusting to coronavirus-related restrictions, and anecdotes about their family, job, or friends.”) In the larger group, commenters cited numbers and percentages more often.

Group chats didn’t make participants more willing to contribute money to Vox. Exactly 12.2% of both volunteer groups — those selected to participate in the group chats and those who weren’t — said they weren’t willing to support Vox financially; the rest responded “yes” or “maybe.”

But among those who were willing to contribute, group chat participants were willing to go deeper in their wallets. About 43% of small group chat participants who answered “yes” or “maybe” said they would be willing to contribute more than $10, compared to just 11% of those not selected to participate in the group chats.

Group chat participants reported a decreased sense of loyalty to The Weeds Facebook group. The Center for Media Engagement guessed that this was “likely because nearly three-fourths of people found the experience to be better than the large Facebook group experience.” This could also be a result of asking participants to prioritize their group chats over conversations happening in the larger Facebook group.