Here at Nieman Lab, we’ve been following the discussions around the value of the comments sections for years, as newsrooms have moderated them, changed platforms, and gotten rid of them all together.

A study published Thursday by the Center for Media Engagement (CME) at the University of Texas, Austin, looked at what comments sections mean to news sites. It found that, for the most part, the comments became less toxic after switching to Coral, the commenting and community platform that Vox acquired from Mozilla in 2019.

It answers the question: What is the value of news comments? We find that removing comments reduces average time on site and that comment toxicity and commenting behavior vary depending on the platform. 2/5

— Talia Stroud (@TaliaStroud) October 22, 2020

Last December, CME worked with 24 Gannett newsrooms, all of which were had been using Facebook Comments on their sites, and randomly assigned each to do one of the following for a six-week period: Turn off their comments section entirely; continue using Facebook Comments on their site; switch to Coral’s commenting system, which requires registration; or switch to Coral’s commenting system but only let subscribers comment.

“The commenting platform that was used did affect the level of toxicity,” according to the study’s authors:

“Overall, sites that adopted Coral had less toxic comments than sites that retained Facebook comments. Averaging by site, sites that switched to Coral for Registrants had 0.05 less toxic comments, sites that switched to Coral for Subscribers had 0.04 less toxic comments, and sites that remained with Facebook had 0.004 less toxic comments. This is not to say, however, that all sites that switched to Coral had less toxic comments (e.g., the Detroit Free Press had slightly more toxic comments). Rather, the aggregate effect across sites is that Coral resulted in fewer toxic comments than Facebook.”

While comments were less toxic on Coral, the commenters on sites using it reported “feeling more disconnected from other commenters, possibly due to people having new screen names.” However, engagement with the comments section increased when Coral for Registrants was used. The study also found that people spent less time on news sites with no comments section at all, compared to sites with Facebook Comments.

“Average time on site declined for all sites participating in this study,” the report’s authors write. While “one theory about eliminating comment sections on a news site is that it will drive people to social media,” that didn’t happen in this study, at least not on the news sites’ Facebook pages (the study didn’t look at Twitter):

“There was no evidence that people commented more frequently on the news organization’s social media page when comments were turned off on the website. The number of comments left each day on the sites’ Facebook pages after the intervention was similar whether the actual news website continued to use the Facebook commenting tool or had no commenting available.”

Read the full study here.