Twitter  This Week in Review: Making sense of a Fox/Time Warner deal, and the future of NYT digital subscription products nie.mn/1ljlOPH  
Nieman Journalism Lab
Pushing to the future of journalism — A project of the Nieman Foundation at Harvard

How does a country get to open data? What Taiwan can teach us about the evolution of access

Assumptions about government openness vary from country to country. Here are a few lessons a cross-national perspective can bring to the open data movement.

TAIPEI — I recently had the pleasure of teaching a two-day data journalism workshop at the Commonwealth Magazine Group in Taiwan’s capital city. The signature moment came when I showed these journalists the L.A. Times crime map, and explained that it updated automatically from government data feeds.

There were gasps from my audience. “Why does the government publish all of that crime data, online, for free?” they asked.

taiwan-data-journalism-workshop

Commonwealth Magazine data journalism workshop in Taipei, Taiwan. Note bubble tea in foreground.

I realized I didn’t have a quick answer to that. In the United States, in 2013, it’s widely assumed that governments on all levels should make their data available for public use. But why? How did we get here? And, importantly, how do other countries get there?

Since then I’ve been trying to understand the history of open data in the U.S. The underlying right to data access comes from the 1966 Freedom of Information Act, and a 1996 update which requires that frequently requested information be made available online. Less obvious, but no less important: American journalists have long been agressive in asking, cajoling, or suing to get access to data. It is the combination of enlightened laws plus a long history of people claiming their rights under these laws that makes transparency seem so natural now. There are entire institutions devoted to the concept, such as the Sunlight Foundation.

Taiwan has a different history. The island nation has enjoyed a relatively open media environment since the end of martial law in 1987, and today it ranks 47th in Reporters Without Borders’ press freedom index, the highest in Asia. (The U.S. ranks 32nd.) There is a fiercely competitive ecosystem of non-state media organizations. Though there have been cases of censorship in the last decade, Commonwealth Magazine managing editor Yin-Chen Wu said government interference is never a concern for her.

There is also a Freedom of Government Information law, enacted in 2005. During the workshop, we talked about how to obtain data, and I asked if anyone had ever used this law.

“We have a freedom of information law?” was the answer. I had discovered it during my research for the workshop, but just because a law is the books doesn’t mean anyone knows about or uses it. Also, under Taiwan’s civil law system, there isn’t any case law which tells the courts how to interpret this right. A few veteran reporters did know of the law, but weren’t aware of anyone who’d ever filed a request under its provisions.

It’s not that the journalists at Commonwealth Magazine don’t want the data. Indeed, the magazine — sometimes described as the Taiwanese equivalent of The Economist — is an Asian data journalism pioneer, winning awards for its infographics and now moving into interactives. Neither are they politically constrained in what they can cover or how they can cover it. Rather, the difficulty is access. “We can write about anything, but there is this problem of getting correct information from the source,” Wu said.

Taiwan’s open data community

Although the Freedom of Government Information law is not much used, there are other ways to get data in Taiwan, and a young open data community is advocating for access. T.H. Schee runs the IT consulting company Fertta.com and organizes many of Taiwan’s open data events. He has been keeping a spreadsheet of open data events since 2008, which now lists over 160 meetups, workshops, and hackathons.

The open data community is a diverse group of several hundred people, including technologists, activists, and domain experts from industry and government. Schee worked for the government himself from 2006 to 2009, at the Academia Sinica research institute, and said that the government experience within the community is part of what makes it successful. “And the program actually works,” he said, “because the government launched several data portals and they are more willing to talk to us.”

There are now five cities in Taiwan with open data portals. Schee’s talk at the W3 eGovernment interest group meeting in January describes the process of getting to this point.

th-schee-open-journalism-taiwan

The progression of Taiwan’s open data ecosystem, from a talk by T.H. Schee.

Public data in private hands

It’s not always the public sector that owns public data, especially in Taiwan.

According to Schee, there are only about 55 IT staff supporting the city of Taipei’s 30,000 public servants. Hence, most of the city’s IT functions are outsourced to private companies. These companies create and manage the majority of public data, and are often reluctant to release it online for free.

Tired of waiting, some people have published privately held government data without official permission. Power generation has been a hot issue in Taiwan since the Japanese reactor meltdowns in the wake of the 2011 tsunami, but official data on the capacity, safety, and radiation monitoring of Taiwan’s nuclear plants has been hard to come by. A small group of industry experts had this data, but lacked any official path to release it. So, they came to the Taipei Open Data Day hackathon and released it in a Dropbox folder.

Schee acknowledges that this act falls into a “gray area,” but notes that “the people who released it did not get in trouble. They have been in the industry for 20 years — they know the legal and ethical boundaries.” Yet simply releasing the data was not enough; they needed to get it to people who would use it. “The experts who released it had a hard time gathering attention among the young hackers and kids, so they came to our hackathon,” said Schee.

Meanwhile, many companies with public data have quietly put it online voluntarily. “The current problem is that the data is available, but it is rather fractured. If you want to write an article, it is very hard and labor intensive to find all the good data,” Schee said. To try to solve this problem, the open data community has created an extensive list of data sources in Taiwan.

Commonwealth Magazine journalists told me they usually obtain data by asking their contacts in government. There has long been data — but publicly accessible, well-organized, free data is a relatively new concept. Schee believes that the ultimate solution to the problem of public data in private hands lies in changes to procurement laws. “For example, if one company is doing map work for the government, they should release it. For this to happen, the procurement law would have to require this. “

Different culture, different sensitivities

In Taiwan, it’s easy to get certain types of information that might horrify Americans, and hard to get other data that Americans would deem obviously public.

“In Taiwan, you would not succeed in getting crime data on the web,” said Schee. “People feel it’s an ethical issue, that you should not put crime data on the web.” Yet images from Taiwan’s huge number of surveillance cameras are deemed less sensitive. “People here are very comfortable being filmed by road cameras, and journalists can get an image from a road camera just by asking the police.”

The furor over Taipei real estate prices illustrates another difference. Taipai housing prices have soared in the last few years, but until very recently transaction prices were a closely held industry secret. That changed at the end of 2012, with new regulations requiring brokers to post selling prices to a central online registry. But the resulting online system only displayed the price of one transaction at a time, so an enterprising group of four engineers took it upon themselves to scrape the entire database and produce a map of real estate prices in Taipei.

The online app got national attention, including print and TV reports. At its peak, over a million people viewed it a day. The app was built using commodity technology that will be familiar to news application developers, such as Open Layers maps and Heroku servers. Visitors found it far more usable and friendly than the official government app.

American data journalists might not imagine that real estate prices could be such a controversial subject, but the context is different in Taiwan. “By creating a layer of intransparency, [real estate brokers] can have much more margin on each transaction,” said Schee.

Lessons for opening data

Comparing across cultures reveals your assumptions, and I think there are several important lessons in the similarities and differences between the American and Taiwan open data stories.

  • Open data is not only about laws, but the ways people use them. It’s as much about culture and expectations as it is about legislation and technology. Journalists and others must advocate for a culture of access.
  • Both top-down and bottom-up efforts are necessary: Governments have a key role in changing laws and publishing data, but independent projects under the mantra of “proceed until apprehended” can be faster, cheaper, and far more radical than official efforts.
  • Different data will be controversial in different times and places. Don’t assume that because something is obviously public or private where you come from, it’s the same somewhere else.
  • You can’t have a discussion about open data without also having a discussion about privacy. When is privacy a legitimate right that we must all protect, and when is it trumped by public interest?
  • As the public data ecosystem develops, journalists will transition from “my source gave me the data” to “I downloaded it.” But you have to find the data before you can download it, which means centralized indices play a key role. These could be official data portals, independent data search engines, or DIY lists of published data.

These lessons might apply not just to countries, but at any level of the open data ecosystem, from the local to the international.

I take one more lesson from my experiences in Taiwan, about the connection between open data movements and data journalists. At its best, the open data movement can support new forms of accountability, and enable innovative new businesses. But to be a part of this, journalists will have to understand and support open data. Even in the U.S., journalists have not always been aware of or involved in the diverse open data community, a community that includes many of the same government agencies that journalists are professionally obliged to suspect.

In Taiwan, the nascent data journalism community is just beginning to engage the broader open data community. “The data journalists are not yet connected to the diverse communities here,” Schee said. But he notes that there were four journalists at the most recent open data hackathon — an encouraging sign.

                                   
What to read next
BuzzFeed mobile
Caroline O'Donovan    
Ben Smith says it will be a guide through the noise of news in social media, and that journalists should realize they’re “part of a collaborative project with other news outlets.”
  • http://twitter.com/scheeinfo TH Schee

    This is a great wrap-up of the path that we’ve taken and thanks Jonathan for his inquisitive style. Journalism and media is at the center of our approach because a healthy and strong narrative culture is much needed to push the agenda forward. We’re looking to share more in days to come to a wider audience.

  • http://twitter.com/WinklerPartners Winkler Partners

    Some additional comments on the legal framework for freedom of information in Taiwan are here along with some statistics for government action on requests for information and brief case notes.

  • http://twitter.com/WinklerPartners Winkler Partners