Nieman Foundation at Harvard
HOME
          
LATEST STORY
This is how The New York Times is using bots to create more one-to-one experiences with readers
ABOUT                    SUBSCRIBE
April 4, 2016, 2:04 p.m.
Reporting & Production
LINK: panamapapers.icij.org  ➚   |   Posted by: Ricardo Bilton   |   April 4, 2016

Sunday’s report of the Panama Papers detailed a global, decades-long story of bribery, arms deals, tax evasion, and financial fraud that implicates over 100 politicians and public officials. Equally impressive, however, is the project itself, which was kept secret for nearly two years despite the involvement of hundreds of reporters.

Here are some details about how the project came together.

It was indeed a major leak. The entire document set, which consists of data going back 40 years, came in at 2.6 terabytes — the largest that reporters have ever had to work with. It’s a thousand times larger than even the 2010 WikiLeaks Cablegate leak, totaled a mere 1.73 gigabytes. In total, the Panama Papers leak consisted 11.5 million files, including 4.8 million emails, 1 million images, and 2.1 million PDFs.

panama-papers2

The process was high tech. The process started way back in 2014, when an unknown source contacted a reporter at German newspaper Süddeutsche Zeitung via encrypted chat, as Wired reports. Later, to process the millions of leaked, developers from The International Consortium of Investigative Journalists built a custom search engine, which the group shared with partner news organizations around the world. The effort also included a real-time chat system for reporters to talk to each others, algorithms that searched for links between names mentioned in the leaked documents, and software that converted raw data into digital formats.

It was a group effort. While Süddeutsche Zeitung was the initial recipient of the leaked documents, the newspaper didn’t analyze them alone. Working with the ICIJ, the organization tapped teams at France’s Le Monde, Argentina’s La Nación, Switzerland’s Sonntagszeitung, and the U.K.’s Guardian and BBC. In total, over 400 reporters across organizations worked on the project.

Few U.S. outlets were involved. While over 100 news organizations across the world analyzed the Panama Papers, besides the Washington D.C.-based ICIJ, just a small handful of the partner organizations were based in the United States: McClatchy’s Charlotte Observer and Miami Herald, Univisión, and Fusion. (Columbia professor Giannina Segnini is also working with the documents, along with five of her j-school students.) None of The New York Times, The Washington Post, and The Wall Street Journal featured coverage of the Panama Papers on their front pages.

The ICIJ doesn’t plan to release the full set of documents. Unlike WikiLeaks, which dumped its entire set of Cablegate documents on the web for anyone to see, the ICIJ says it doesn’t plan to do the same with the Panama Papers. “We’re not WikiLeaks. We’re trying to show that journalism can be done responsibly,” ICIJ director Gerard Ryle told Wired.

Show tags Show comments / Leave a comment
 
Join the 35,000 who get the freshest future-of-journalism news in our daily email.
This is how The New York Times is using bots to create more one-to-one experiences with readers
“I’m not worried about this technology driving the humanity out of journalism. I’m really excited about the promise of technology bringing more humanity to journalism.” Also: a Michael Barbaro bot.
These are the bots powering Jeff Bezos’ Washington Post efforts to build a modern digital newspaper
“It’s this great, simple experience, and the technology is getting so much better for it: AI’s getting better. big data’s more accessible.” Also: a Marty Baron bot.
The Information’s new Briefing is a continuous update of opinionated takes on other people’s articles
Briefing is meant to be more Politico Playbook than Techmeme. It’s updated around the clock, but is also being sent out as a daily email newsletter for subscribers.