The primary source in the age of mechanical multiplication

“Many stories lay dormant in the vast amounts of data produced by everyday consumers because journalists are still only starting to acquire the large-scale data-wrangling expertise needed to tap them.”

Historians and journalists alike have long prized one source of information above all others: the on-the-record, primary source. They scour the attics for diaries and journals and fly across the country for interviews. But now we have a glut of documentation.

lam-thuy-voThanks to social media, millions of people go on the record, publicly, every single day. People sent billions of tweets, Facebook posts, and WhatsApp messages last year. They have expressed wonder 😮, anger 😤, and love ❤️ for social and political issues.

At present, most journalists treat social sources like they would any other — individual anecdotes and single points of contact. But to do so with a handful of tweets and Instagram posts is to ignore the potential of hundreds of millions of others.

Many stories lay dormant in the vast amounts of data produced by everyday consumers because journalists are still only starting to acquire the large-scale data-wrangling expertise needed to tap them. As more and more people conduct their lives online, and as smartphones are penetrating previously unconnected regions around the world, this trove of stories is only becoming larger.

The kinds of stories journalists can tell using this data are wide ranging. We can reconstruct online encounters in ways more precise than a source may recall from memory. Le Monde, for instance, retraced the journey of Syrian refugees through their WhatsApp messages. At Al Jazeera America, we analyzed and chronicled the evolution of a Hong Kong pro-democracy movement in Facebook chatrooms.

Journalists can try to find insight to people’s personality and character or hold powerful people accountable. Data scientist David Robinson did a sentiment analysis of Donald Trump’s tweets and found that Trump’s own tweets are much more negative than those his campaign staff tweeted. My colleague Charlie Warzel and I looked at the links Trump tweeted to explore the news he chooses to circulate, as a proxy for the news he may consume.

Journalists can examine the ways in which technology disadvantages groups by looking at social data. ProPublica’s Julia Angwin and Terry Parris Jr. bought a Facebook ad and established that the social media company allows advertisers to exclude customers based on race, while Vox’s Alvin Chang expanded on ProPublica’s analysis by looking at whether Facebook’s algorithm excludes already disadvantaged populations from being offered opportunities that their more affluent counterparts receive.

When journalists venture into this kind of story mining, I hope that they also continue to discuss the ethics surrounding it, the blurred lines between what is considered public and what is considered private, and the caveats that come with each dataset.

Last but not least, I hope that journalists will dig into social data to gain insights into whom they reach and, perhaps more importantly, whom they do not reach. People live in their filtered worlds in which algorithms serve them information that tends to affirm rather than question political views.

Maybe social data can allow us to understand these bubbles better in an effort to pierce them.

Lam Thuy Vo is a fellow in BuzzFeed’s Open Lab for Journalism, Technology, and the Arts.

Caitlin Thompson   High touch, high value

Nushin Rashidian   A rise in high-price, high-value subscriptions

Swati Sharma   Failing diversity is failing journalism

Rasmus Kleis Nielsen   News after advertising may look like news before advertising

Rebekah Monson   Journalism is community-as-a-service

Rubina Madan Fillion   Snapchat grows up

Mike Ragsdale   A smarter information diet

Tanya Cordrey   The resurgence of reach

Erin Pettigrew   A year of reflection in tech

Bill Adair   The year of the fact-checking bot

Millie Tran   International expansion without colonial overtones

Sam Ford   The year we talk about our awful metrics

Jim Friedlich   A banner year for venture philanthropy

Mira Lowe   News literacy, bias, and “Hamilton”

Mary Meehan   Feeling blue in a red state

Amie Ferris-Rotman   Вслед за Россией

Mark Armstrong   Time to pay up

Guy Raz   Inspiration and hope will matter more than ever

Doris Truong   Connecting with diverse perspectives

Jonathan Hunt   Measurement companies get with the times

Margarita Noriega   From pinning tweets to tweeting pins

Ashley C. Woods   Local journalism will fight a new fight

Ryan McCarthy   Platforms grow up or grow more toxic

Michael Oreskes   Reversing the erosion of democracy

Libby Bawcombe   Kids board the podcast train

Sara M. Watson   There is no neutral interface

Errin Haines Whack   Chaos or community?

Coleen O'Lear   Back to basics

Trushar Barot   API or die

Andrew Losowsky   Building our own communities

Molly de Aguiar   Philanthropists galvanize around news

Annemarie Dooling   UGC as a path out of the bubble

P. Kim Bui   The year journalism teaches again

Megan H. Chan   Cultural reporting goes mainstream

Joanne Lipman   The year of the drone, really

Olivia Ma   The year collaboration beats competition

Renée Kaplan   Pure reach has reached its limit

Sue Schardt   Objectivity, fairness, balance, and love

Michael Kuntz   Trust is the new click

Mary Walter-Brown   Getting comfortable asking for money

Mandy Velez   The audience is the source and the story

David Chavern   Fake news gets solved

Laura Walker   Authentic voices, not fake news

Carla Zanoni   Prioritizing emotional health

Liz Danzico   The triumph of the small

Eric Nuzum   Podcasting stratifies into hard layers

Gabriel Snyder   The aberration of 20th-century journalism

Keren Goldshlager   Defining a focus, and then saying no

Peter Sterne   A dangerous anti-press mix

Julia Beizer   Building a coherent core identity

Alice Antheaume   A new test for French media

Jeremy Barr   A terrible year for Tiers B through D

Tim Herrera   The safe space of service journalism

Ariane Bernard   Better data about your users

Almar Latour   Thanks, #fakenews

Ray Soto   VR moves from experiments to immersion

Hillary Frey   Forests need to burn to regrow

Ernst-Jan Pfauth   Earn trust by working for (and with) readers

Taylor Lorenz   “Selfie journalism” becomes a thing

Pablo Boczkowski   Fake news and the future of journalism

Tressie McMillan Cottom   A path through the media’s coming legitimacy crisis

Steve Henn   The next revolution is voice

Scott Dodd   Nonprofits team up for impact

Tracie Powell   Building reader relationships

Jonathan Stray   A boom in responsible conservative media

Dan Gillmor   Fix the demand side of news too

Rachel Schallom   Stop flying over the flyover states

Aja Bogdanoff   Comments start pulling their weight

Helen Havlak   Chasing mobile search results

David Weigel   A test for online speech

Cory Haik   Navigating power in Trump’s America

Ole Reißmann   Un-faking the news

Valérie Bélair-Gagnon   Truthiness in private spaces

Moreno Cruz Osório   The year of transparency in Brazilian journalism

Emily Goligoski   Incorporating audience feedback at scale

Claire Wardle   Verification takes center stage

An Xiao Mina   2017 is for the attention innovators

Tim Griggs   The year we stop taking sides

Juliette De Maeyer and Dominique Trudel   A rebirth of populist journalism

Kawandeep Virdee   Moving deeper than the machine of clicks

Umbreen Bhatti   A sense of journalists’ humanity

S.P. Sullivan   Baking transparency into our routines

Matt Waite   The people running the media are the problem

Francesco Marconi   The year of augmented writing

Katie Zhu   The year of minority media

Mathew Ingram   The Faustian Facebook dance continues

Richard J. Tofel   The country doesn’t trust us — but they do believe us

Lee Glendinning   A call for great editing

Andy Rossback   The year of the user

David Skok   What lies beyond paywalls

Ken Schwencke   Disaggregation and collection

Dan Colarusso   Let’s make live video we can love

Priya Ganapati   Mobile websites are ready for reinvention

Vivian Schiller   Tested like never before

Bill Keller   A healthy skepticism about data

Sarah Wolozin   Virtual reality on the open web

Jon Slade   Trusted news, at a premium

Alexis Lloyd   Public trust for private realities

Samantha Barry   Messaging apps go mainstream

Amy O'Leary   Not just covering communities, reaching them

Alberto Cairo   Communicating uncertainty to our readers

Elizabeth Jensen   Trust depends on the details

Adam Thomas   The coming collaboration across Europe

Amy Webb   Journalism as a service

Geetika Rudra   Journalism is community

Nicholas Quah   Podcasting’s coming class war

Sarah Marshall   Focusing on the why of the click

Juan Luis Sánchez   Your predictions are our present

Robert Hernandez   History will exclude you, again

Melody Kramer   Radically rethinking design

Kathleen Kingsbury   Print as a premium offering

Emi Kolawole   From empathy to community

Nathalie Malinarich   Making it easy

Dhiya Kuriakose   The year of digital detoxing

Ståle Grut   The battle for high-quality VR

Rachel Sklar   Women are going to get loud

Reyhan Harmanci   Bear witness — but then what?

Anita Zielina   The sales funnel reaches (and changes) the newsroom

Erin Millar   The bottom falls out of Canadian media

Laura E. Davis   Show your work

Dannagal G. Young   The return of the gatekeepers

Matt Karolian   AI improves publishing

Zizi Papacharissi   Distracted journalism looks in the mirror

Carrie Brown-Smith   We won’t do enough

Andrew Haeg   The year of listening

Javaun Moradi   What can we own?

Liz McMillen   The year of deep insights

Maria Bustillos   “It’s true — I saw it on Facebook”

Mario García   Virtual reality on mobile leaps forward

Andrea Silenzi   Podcasts dive into breaking news analysis

Felix Salmon   Headlines matter

Andrew Ramsammy   Rise of the rebel journalist

Corey Ford   The year of the rebelpreneur

Lam Thuy Vo   The primary source in the age of mechanical multiplication

Burt Herman   Local news gets interesting

Sydette Harry   Facing journalism’s history

Cindy Royal   Preparing the digital educator-scholar hybrid

Asma Khalid   The year of the newsy podcast

Christopher Meighan   Unlocking a deeper mobile experience

M. Scott Havens   Quality advertising to pair with quality content