Nieman Foundation at Harvard
HOME
          
LATEST STORY
Dow Jones negotiates AI usage agreements with nearly 4,000 news publishers
ABOUT                    SUBSCRIBE
Feb. 25, 2019, 11:40 a.m.
Audience & Social
LINK: www.theverge.com  ➚   |   Posted by: Laura Hazard Owen   |   February 25, 2019

Casey Newton’s unprecedented investigation into the contract workers who moderate content on Facebook in America is full with horrifying, dystopian tidbits. Horrifying like: Workers are getting PTSD from repeatedly watching videos of murders in which the victims cry for their mothers as they die.

The Verge has summarized some of Newton’s findings in a TL;DR sidebar (for instance, “Employees can be fired after making just a handful of errors a week, and those who remain live in fear of former colleagues returning to seek vengeance. One man we spoke with started bringing a gun to work to protect himself”). But here I’ll focus on just one aspect: the fact that some of these content moderators are becoming radicalized themselves, beginning to believe the conspiracy theories that they are paid to flag. From the piece:

The moderators told me it’s a place where the conspiracy videos and memes that they see each day gradually lead them to embrace fringe views. One auditor walks the floor promoting the idea that the Earth is flat. A former employee told me he has begun to question certain aspects of the Holocaust. Another former employee, who told me he has mapped every escape route out of his house and sleeps with a gun at his side, said: “I no longer believe 9/11 was a terrorist attack.”

And:

One [quality assurance manager] often discussed his belief that the Earth is flat with colleagues, and “was actively trying to recruit other people” into believing, another moderator told me. One of Miguel’s colleagues once referred casually to “the Holohoax,” in what Miguel took as a signal that the man was a Holocaust denier.

Conspiracy theories were often well received on the production floor, six moderators told me. After the Parkland shooting last year, moderators were initially horrified by the attacks. But as more conspiracy content was posted to Facebook and Instagram, some of Chloe’s colleagues began expressing doubts.

“People really started to believe these posts they were supposed to be moderating,” she says. “They were saying, ‘Oh gosh, they weren’t really there. Look at this CNN video of David Hogg — he’s too old to be in school.’ People started Googling things instead of doing their jobs and looking into conspiracy theories about them. We were like, ‘Guys, no, this is the crazy stuff we’re supposed to be moderating. What are you doing?’”

This phenomenon is not limited to the contract workers of Cognizant, the firm Facebook outsources moderation to; it is a known problem for people who spend a lot of time studying conspiracy theories and disinformation online. Kate Starbird, an assistant professor at the University of Washington and director of its Emerging Capacities of Mass Participation (emCOMP) Laboratory, spoke about this at Harvard last year. “When you do this kind of thing, you should keep a rope around your ankle and have someone to pull you back up,” she said. “This is a really disorienting part of the Internet.” People on her team found themselves beginning to believe some of the conspiracy theories that they were reading about; it was something everybody had to be vigilant about.

Starbird spoke about the effects — on her, her research partners, and students — of reading so much of this kind of content. It’s “very effective and extremely disorienting,” she said. “We’ve had trouble even reporting on this data because we’re so confused about what’s going on and it’s so hard to make sense of things…This way of thinking, once you get into it, is very sticky. Once you’re down there, it’s so hard to get out, and it’s hard to think of solutions.”

While Cognizant provides some meager mental health support to its employees — and, uh, nine minutes per day of “wellness time” for when you get too traumatized, which Muslim employees were not allowed to use to pray — neither it nor Facebook appear to be addressing the concern that this work is generating a new crop of conspiracy theorists — the moderators themselves.

Read the full piece here.

Show tags
 
Join the 60,000 who get the freshest future-of-journalism news in our daily email.
Dow Jones negotiates AI usage agreements with nearly 4,000 news publishers
Earlier this year, the WSJ owner sued Perplexity for failing to properly license its content. Now its research tool Factiva has negotiated its own AI licensing deals.
Back to the bundle
“If media companies can’t figure out how to be the bundlers, other layers of the ecosystem — telecoms, devices, social platforms — will.”
Religious-sounding language will be everywhere in 2025
“A great deal of language that looks a lot like Christian Nationalism isn’t actually calling for theocracy; it is secular minoritarianism pushed by secular people, often linked to rightwing cable and other media with zero meaningful ties to the church or theological principle.”