2
0
1
9

More algorithmic accountability reporting, and a lot of it will be meh

“When reporters find a suspect algorithm, they should also try to cover what could be done better.”

We’ve become suspicious of tech. Once obscure issues like the design of news recommendation systems and the evaluation of criminal justice risk scoring methods have become major public conversations. This new scrutiny of technical systems with social impact is well deserved — but that doesn’t mean that we get to be sloppy when investigating the power of algorithms.

The journalistic investigation of the politics of code, sometimes called algorithmic accountability reporting, is one of the more complicated types of reporting to do. The interactions between the technical and the social are intricate. Covering California’s SB 10 bill, which mandates the use of  pre-trial statistical risk assessment, requires an understanding of both machine learning error rates and the contentious politics of bail reform. The New York Times tried to make the case that increased social media use in Germany is correlated with more violent attacks on refugees, but the “landmark” study they relied on was actually a preliminary paper. Subsequent analysis suggests that the truth might end up turning on the number of people in the Facebook Nutella group, which the researchers used as a proxy for social media use generally. This stuff is tricky — and because the stakes are so high, everyone has strong opinions.

I’ve already seen several cringe-worthy examples of simplistic, unfounded, or just plain biased reporting on algorithms, including misleading pieces from both tech cheerleaders and tech skeptics. In the hopes of seeing better work in the future, here are a few tips on getting an algorithm story right.

  • Method matters. Investigating the social consequences of algorithmic decision-making is an act of social science. Don’t let that scare you — data journalists have been doing social science since the beginning. But it does mean there’s a lot to learn from other disciplines about analyzing data, designing experiments, and so on. The causes of political events are especially difficult to pin down, yet that’s exactly what much algorithmic accountability reporting aims to do. Plus, social scientists have invented lots of creative methods that journalists can use. Here’s a great primer on how to design experiments to reverse engineer social media platforms.
  • Interdisciplinary details matter. There aren’t many people who are equally knowledgeable about both the technical and the social. Writing on algorithms seems to split down this divide: We get detailed technical analyses of machine learning algorithms or we get sophisticated legal arguments about the consequences of using algorithms in court, but very little that integrates these perspectives. And what about community members and advocates, the people most affected by these algorithms? Journalists have to straddle these schisms. This is a strong argument for teamwork and collaboration between people with different backgrounds.
  • Transparency matters. A great many algorithms of social interest are black boxes: we can’t see the code, and often we can’t even interact with the system to test it. There are a variety of ways around this, including crowdsourcing to collect algorithmic results. But sometimes you’re just not going to be able to get the information you want. In this case, as the old saying goes, the fact that you can’t do the story becomes the story. Why isn’t it possible for an independent person to check on the function of critical societal infrastructure? What would have to change to enable this sort journalism?
  • What’s the option? I’m going to borrow a page from the Solutions Journalism movement and suggest that when reporters find a suspect algorithm, they also try to cover what could be done better. Sometimes the algorithm should not be used. Sometimes there are already good ideas about how to modify it. And sometimes there is evidence that human decisions are worse than statistical methods. Kentucky judges routinely ignore algorithmic risk assessments and jail people who are classified as low risk, while automated lending systems have reduced discrimination.

Like any serious journalism, algorithmic accountability reporting requires expertise, curiosity, and dedication to the truth. Increased skepticism of our robot overlords is a good thing, but it doesn’t get to play by different standards than any other investigative journalism.

Jonathan Stray is a computational journalist teaching and researching at Columbia University’s School of Journalism.

Angilee Shah   The year news orgs say “yes” to real leaders

Ole Reißmann   The rise of vertical storytelling

Soo Oh   Just showing our work isn’t enough

Brian Moritz   The subscription-pocalypse is about to hit

Robert Hernandez   Racists and sexists get replaced

Mike Caulfield   Ditch the media literacy cynicism and get to work

Tyler Fisher   This is journalism’s do-or-die moment

Tamar Charney   Seriously: What do you do for people?

Linda Solomon Wood   The year of the climate reporter

Kelsey Proud   Journalism becomes the escape

Cindy Royal   For journalism curriculum to change, its faculty needs disruption

Michael Grant   More newsrooms experiment their way to success

Masuma Ahuja   Make foreign coverage less foreign

Sarah Marshall   A return to destination journalism

Rubina Madan Fillion   Fighting the reality of deepfakes

Marie Shanahan   Newsrooms take the comments sections back from platforms

Annie Rudd   A more intimate aesthetic of politics — on Insta

Renée Kaplan   Our future could lie within our own organizations

Almar Latour   Reported facts, weaponized in service of action

Renan Borelli   Developing loyalty means developing your talent

John Biewen   Podcasts keep getting better

Jack Riley   Facebook refugees, from ad revenue to news habits

Celeste LeCompte   Local news needs local conversation to survive

Kawandeep Virdee   Media wants to take care of you

Bill Adair   Another year fighting Trump’s falsehoods

Simon Galperin   After capitalism’s fire, journalism’s secondary succession

Frank Mungeam   Tonight at 11: News, sports, and climate change

Cherian George   Fake news wins in Asia

Gabriel Snyder   Journalism doesn’t fit well in a funnel

Ariel Zirulnick   Participation gets professional

Sue Robinson   Reporters go on the offensive

j. Siguru Wahutu   Think 2018 was bad? Wait until you see 2019

Becca Aaronson   From bridge roles to product thinkers

Joshua P. Darr   The nationalization of political news will accelerate

Julia Rubin   Meeting people where they are

Libby Bawcombe   Haikus of the news

Stephanie Edgerly   It’s time to understand the un-audience

Simon Rogers   Data journalism becomes a global field

Millie Tran   There is no magic — you’ve got this

Dave Burdick   Seeing our blind spots

Whitney Phillips   Our information systems aren’t broken — they’re working as intended

Jenée Desmond-Harris   It finally sinks in that some people aren’t white

Matt Karolian   Publishers come to terms with being Facebook’s enablers

Alberto Cairo   A year of uncertainty and confidence

Axie Navas   The traffic hunt, CMS battle, and magazine identity crises loom

Mat Yurow   Content competition from the tech companies

Victor Pickard   We will finally confront systemic market failure

Salem Solomon   Correcting our corrections

Reyhan Harmanci   Selling more stories to Hollywood

P. Kim Bui   The misfits become the bosses

Rachel Glickhouse   Newsrooms will prioritize audience needs

Rasmus Kleis Nielsen   A long, slow slog, with no one coming to the rescue

Kjerstin Thorson   Time to get mad about information inequality (again)

Umbreen Bhatti   The story doesn’t end for the people we quote

Jean Friedman Rudovsky   Cross-newsroom collaborations strengthen communities

Alexandra Svokos   Good luck convincing us millennials to pay

Emma Carew Grovum   The year of the loyal reader

Laura E. Davis   More access, but not that kind

Taylor Lorenz   Personal branding is more powerful than ever

Kainaz Amaria   We consider who’s behind the camera

Mariana Moura Santos   From pageviews to impact

Heather Bryant   We are responsible for how we use our power

Michael Rain   The year of the culturally relevant curator

Seth C. Lewis   The gap between journalism and research is too wide

Pablo Boczkowski   Reimagining the media for post-institutional times

Jesse Holcomb   We’ll get better at making the case for local journalism

Rodney Gibbs   A bright — and young — year for audio

Tim Carmody   Unlocking the commons

Frank Chimero   Leave the phone at home and put news on your wrist

Zizi Papacharissi   Old interface, say hello to the new interface

John Saroff   The pivot to reader revenue’s unintended consequences

Bill Grueskin   Toward a symphony model for local news

Sarah Alvarez   Simplify and redistribute

A.J. Bauer   The coming splintering of conservative media

Carolina Guerrero   Spanish-language audio blows up

Tushar Banerjee   Interactive ads will be the new face of display advertising

Nico Gendron   Reaching Generation Z beyond the coasts

Adam B. Ellick   Video forensic reporting goes mainstream — and local

Joe Amditis   Give the audience a seat at the table

Nisha Chittal   The homepage makes a comeback

Andrew Donohue   Voting rights becomes the new climate change

Elizabeth Jensen   Going where the Acela can’t take you

Sarah Stonbely   Mapping the local news ecosystem — with scale but detail

Elisabeth Goodridge   Yes, they signed up — but our job’s not over

Amy Schmitz Weiss   Local news isn’t where you thought it was

Rick Berke   The year of loyalty

Gideon Lichfield   Goodbye attention economy, we’ll miss you

Jim Friedlich   Meet Citizen Kane 2.0

Shannon McGregor   More bogus embedded tweets in our stories

Kevin D. Grant   A year to embrace journalism as public service

Mike Isaac   The old exit doors for digital media companies are closing

Colleen Shalby   Representation becomes more than a talking point

Cory Bergman   Journalism as a technology service

Lauren Katz   Community becomes a core newsroom value

Adam Thomas   In Europe, foundations invest in news

Ben Smith   The pendulum starts to swing back

Cristi Hegranes   A year to invest in the security of local journalists

Elva Ramirez   News — but make it cinematic

Andrew Ramsammy   The great re-pivot to audio

Angèle Christin   Algorithms and the reflexive turn

Josh Schwartz   A pullback from platforms and a focus on product

Jonas Kaiser   Catching up with “Neuland”

Chase Davis   We can acknowledge what we don’t know

Amy King   We should listen to the kids (especially on Instagram)

Francesco Zaffarano   Towards a rethinking of journalism on social media

Kyra Darnton   A shift to depth in video

Pia Frey   You can’t solve a crisis without treating it as a crisis

Rebecca Lee Sanchez   We are all actors in the running rampant of political theater

Alexis Lloyd & Matt Boggie   The year product leads media

Rishad Patel   A design system for responsible publishing

Callie Schweitzer   The rise of the conveners

Hossein Derakhshan   The news is dying, but journalism will not — and should not

Francesco Marconi   The year of iterative journalism

Steve Henn   Smart speakers get smarter

Heather Chaplin   Agree we’re partisan — for the democratic system

Catalina Albeanu   Being responsible for what we don’t know

Alexandra Borchardt   Newsrooms need to build trust with their journalists, not just the audience

Moreno Cruz Osório   Damaged credibility and a new threat in Brazil

Juleyka Lantigua   Podcasting battles East Coast bias

Ståle Grut   A new dawn for 3D tech in journalism

Carrie Brown-Smith   Advocating a healthy civic life is no journalistic crime

Nathalie Malinarich   Video — yes, video

Johannes Klingebiel   We all grow hooves

Hearken   Pivot to people

Knight Foundation   A year of local collaboration

Adam Smith   Platforms will have to help rebuild trust in news

Justin Kosslyn   Text hits a tipping point

Borja Bergareche Sainz de los Terreros   Entering a more balanced era

Elizabeth Bramson-Boudreau   A more sincere definition of “community”

Ruth Palmer and Benjamin Toff   From news fatigue to news avoidance

Ben Werdmuller   The platform tide is turning

Jesse Brown   Canada’s subsidy for news backfires

Meredith Artley   Huge demand for…anything but politics

Zuzanna Ziomecka   News leadership gets an overdue upgrade

Alyssa Zeisler   We expand what (and how and who) we serve

Geetika Rudra   The year of actionable (local) journalism

Matt Skibinski   Quality and reliability are the new currencies for publishers

Peter Bale   Venture capital runs out of patience

John Garrett   You can’t raise prices forever

Elizabeth Dunbar   Local reporters reflect on what’s not important

Mandy Velez   Putting the social back in social media

Dheerja Kaur   A focus on problems, not platforms

Heba Aly   The rise of international nonprofit news

Jennifer Dargan   You don’t build diversity through one-off training sessions

Rachel Davis Mersey   Local news goes minimalist

Seema Yasmin   We will create our own spaces

Eric Nuzum   The year of the DIY podcast network

Elite Truong   What do we owe the next generation?

Darryl Holliday   Let’s talk about power (yours)

Thomas Hanitzsch   The rise of tribal journalism

Betsy O'Donovan and Melody Kramer   The most beautiful sentence in 2019 is “No.”

Nicholas Jackson   More transparency around newsroom decisions

Jeff Chin   We detox from Chartbeat

Monique Judge   Committing to the truth, calling out lies

Jake Shapiro   Podcasting is media’s slow food movement

Dan Shanoff   Bet on sports gambling

Greg Emerson   Power to the user

Robin Kwong   Tech shouldn’t be the only field pollinating “news nerds”

Mandy Jenkins   Fight the urge to run away from social media

Raney Aronson-Rath   We learn “digital” doesn’t have to mean “short”

Peter Cunliffe-Jones   The focus of misinformation debates shifts south

Manoush Zomorodi   Tech will do for information overload what it did for mindfulness

Christa Scharfenberg and Vickie Baranetsky   The year of the lawsuit

Ernst-Jan Pfauth   Readers are only getting started

Stefanie Murray   Local news wakes up and starts collaborating

Carl Bialik   Fatigued news consumers will pay more for less news

Eric Ulken   The year you actually start to like your CMS

Rebecca Searles   From silos to Swiss Army knife teams

Kate Myers   Journalism continues to be bad for democracy

Nikki Usher   Three ways national media will further undermine trust

Joel Konopo   Influencers become the new liberated power in Africa

Matthew Pressman   The battle over objectivity intensifies

Craig Newmark   The end of “loudspeakers for liars”

Don Day   Timewalls and other reader revenue experiments

Candis Callison   Learn from Indigenous journalists on covering climate change

Ernie Smith   The year we step back from the platform

Jonathan Gill   Publishers build a common tech platform together

Patrick Butler   Measuring impact will increase audience trust

Winny de Jong   Data journalism goes undercover

Julie Posetti   The year of the fight back

Mike Rispoli and Craig Aaron   Government funds local news — and that’s a good thing

Talia Stroud   Engaging people across lines of difference

Steve Myers   From trying to cover it all to covering what matters

Matt Waite   “I went to Node.js because I wished to live deliberately”

Tshepo Tshabalala   Ahead of African elections, unlock partnerships with fact-checkers

Charo Henríquez   Pivot to journalism

Sue Cross   Return of the water cooler

Logan Molyneux   Seeing social media for what it is

Glyn Mottershead and Martin Chorley   When a tech company pulls the plug on your story

Joanne McNeil   Building a digital hospice

LaToya Drake   Listen up: New stories, new storytellers

Zainab Khan   Publishers whose products can stand up to social media giants will win

Jonathan Stray   More algorithmic accountability reporting, and a lot of it will be meh

Claire Wardle   Forget deepfakes: Misinformation is showing up in our most personal online spaces

Andrea Faye Hart   Doing less harm, not just more good

M. Scott Havens   Time to swing for the fences

Steve Grove   A reckoning for tech’s work with news

Kristen Muller   Local news fails — in a good way

Efrat Nechushtai   Journalism wants to be your friend, not your teacher

Mario García   The rise of content “pilots”

An Xiao Mina   The death of consensus, not the death of truth

Jeremy Gilbert   AI finally becomes helpful

Jared Newman   AI-generated fakes launch a software arms race

Shalabh Upadhyay   A culture clash on India’s growing Internet

Errin Haines   Say it with me: Racism