Twitter  How are local public media stations doing on innovation? A new survey has some answers nie.mn/1ipr7OP  
Nieman Journalism Lab
Pushing to the future of journalism — A project of the Nieman Foundation at Harvard

Word clouds considered harmful

The New York Times senior software architect would like the newest “mullets of the Internet” to go back from whence they came.
Email

In his 2003 novel Pattern Recognition, William Gibson created a character named Cayce Pollard with an unusual psychosomatic affliction: She was allergic to brands. Even the logos on clothing were enough to make her skin crawl, but her worst reactions were triggered by the Michelin Tire mascot, Bibendum.

Although it’s mildly satirical, I can relate to this condition, since I have a similar visceral reaction to word clouds, especially those produced as data visualization for stories.

If you are fortunate enough to have no idea what a word cloud is, here is some background. A word cloud represents word usage in a document by resizing individual words in said document proportionally to how frequently they are used, and then jumbling them into some vaguely artistic arrangement. This technique first originated online in the 1990s as tag clouds (famously described as “the mullets of the Internet“), which were used to display the popularity of keywords in bookmarks.

More recently, a site named Wordle has made it radically simpler to generate such word clouds, ensuring their accelerated use as filler visualization, much to my personal pain.

So what’s so wrong with word clouds, anyway? To understand that, it helps to understand the principles we strive for in data journalism. At The New York Times, we strongly believe that visualization is reporting, with many of the same elements that would make a traditional story effective: a narrative that pares away extraneous information to find a story in the data; context to help the reader understand the basics of the subject; interviewing the data to find its flaws and be sure of our conclusions. Prettiness is a bonus; if it obliterates the ability to read the story of the visualization, it’s not worth adding some wild new visualization style or strange interface.

Of course, word clouds throw all these principles out the window. Here’s an example to illustrate. About six months ago, I had the privilege of giving a talk about how we visualized civilian deaths in the WikiLeaks War Logs at a meeting of the New York City Hacks/Hackers. I wanted my talk to be more than “look what I did!” but also to touch on some key principles of good data journalism. What better way to illustrate these principles than with a foil, a Goofus to my Gallant?

And I found one: the word cloud. Please compare these two visualizations — derived from the same data set — and the differences should be apparent:

I’m sorry to harp on Fast Company in particular here, since I’ve seen this pattern across many news organizations: reporters sidestepping their limited knowledge of the subject material by peering for patterns in a word cloud — like reading tea leaves at the bottom of a cup. What you’re left with is a shoddy visualization that fails all the principles I hold dear.

Every time I see a word cloud presented as insight, I die a little inside.

For starters, word clouds support only the crudest sorts of textual analysis, much like figuring out a protein by getting a count only of its amino acids. This can be wildly misleading; I created a word cloud of Tea Party feelings about Obama, and the two largest words were implausibly “like” and “policy,” mainly because the importuned word “don’t” was automatically excluded. (Fair enough: Such stopwords would otherwise dominate the word clouds.) A phrase or thematic analysis would reach more accurate conclusions. When looking at the word cloud of the War Logs, does the equal sizing of the words “car” and “blast” indicate a large number of reports about car bombs or just many reports about cars or explosions? How do I compare the relative frequency of lesser-used words? Also, doesn’t focusing on the occurrence of specific words instead of concepts or themes miss the fact that different reports about truck bombs might be use the words “truck,” “vehicle,” or even “bongo” (since the Kia Bongo is very popular in Iraq)?

Of course, the biggest problem with word clouds is that they are often applied to situations where textual analysis is not appropriate. One could argue that word clouds make sense when the point is to specifically analyze word usage (though I’d still suggest alternatives), but it’s ludicrous to make sense of a complex topic like the Iraq War by looking only at the words used to describe the events. Don’t confuse signifiers with what they signify.

And what about the readers? Word clouds leave them to figure out the context of the data by themselves. How is the reader to know from this word cloud that LN is a “Local National” or COP is “Combat Outpost” (and not a police officer)? Most interesting data requires some form of translation or explanation to bring the reader quickly up to speed, word clouds provide nothing in that regard.

Visualization is reporting, with many of the same elements that would make a traditional story effective.

Furthermore, where is the narrative? For our visualization, we chose to focus on one narrative out of the many within the Iraq War Logs, and we displayed the data to make that clear. Word clouds, on the other hand, require the reader to squint at them like stereograms until a narrative pops into place. In this case, you can figure out that the Iraq occupation involved a lot of IEDs and explosions. Which is likely news to nobody.

As an example of how this might lead the reader astray, we initially thought we saw surprising and dramatic rise in sectarian violence after the Surge, because of the word “sect” was appearing in many more reports. We soon figured out that what we were seeing had less to do with violence levels and more to do with bureaucracy: the adoption of new Army requirements requiring the reporting of the sect of detainees. Of course, the horrific violence we visualized in Baghdad was sectarian, but this was not something indicated in the text of the reports at the time. If we had visualized the violence in Baghdad as a series of word clouds for each year, we might have thought that the violence was not sectarian at all.

In conclusion: Every time I see a word cloud presented as insight, I die a little inside. Hopefully, by now, you can understand why. But if you are still sadistically inclined enough to make a word cloud of this piece, don’t worry. I’ve got you covered.

Jacob Harris is a senior software architect at The New York Times.

                                   
What to read next
INNlogo_blue
Justin Ellis    April 15, 2014
Chalkbeat, Southern California Public Radio, InvestigateWest and others are awarded over $236,000 in micro-grants to support events programming, collaborative reporting, and a “native underwriting” pilot program.
  • Vladimir Nedovic

    “Every time I see a word cloud presented as insight”

    I think the above is the problem: the author assumes the word cloud to be perceived as an insight into the story, whereas I think (hope) that people do not give it as much importance as his worry implies. It’s just an interesting indicator of the importance given to certain words, and if anybody is going to make any conclusions about the substance from only the word cloud, well, that’s their problem. People who want to get an insight will not assume the word cloud to contain it all, but will dig deeper.

  • http://twitter.com/harrisj Jacob Harris

    Sadly, I wish that were the case, but I usually only see a word cloud presented as a mechanism for visualization and insight. Otherwise, if it’s a filler image, Internet protocol dictates you’re supposed to use a lolcat. For instance, see this http://www.huffingtonpost.com/2011/10/13/occupy-wall-street-word-cloud_n_1009023.html 

  • Darrel

    Pretty much disagree. Smacks of snobbery to me. Plus your site is terribly rendered on a mobile device. Thank goodness I was connected via wifi!

  • http://twitter.com/harrisj Jacob Harris

    Yes, but it’s hard to find snobbery this pure on the street any more. Most of it is usually cut with snark or irony.

  • Shaun Ryan

    your looking word clouds from an incomplete perspective. Obviously if you have numbers with attributes thaen distribution and trends there are better ways to represent the information from the data. If they having relation with geography then perhaps plotting on map will help. However I’ve also seen chronic obsession with mapping data that presents nothing relevant to information being presented other take up large amounts of space. Word clouds are useful where you might have millions of documents or website searches and you want to know at glance what’s being written about or what’s being searched on yor it’s. It more about using the right visualisation for use and data, this is a major problem with data visulisation. Whilst pie charts exist and are used then it’s obvious that Data visualisation doesn’t get the attention it deserves.

  • http://twitter.com/harrisj Jacob Harris

    Admittedly, I chose a contrived example, but I think Word Clouds are flawed even for purely textual analysis. There are much more powerful ways of looking through and analyzing text than the simplistic “bag of words” model used by word clouds. Even using the classic tf*idf model of basic search analysis would be better than just overall term frequency.

  • Moreen

    Word clouds serve a different purpose for a different audience and have utility for the purpose they are designed to serve, just like your map.  That said, you are right to caution against the misrepresentation of word clouds as decontextualized insight.

    In context, used by writers and speakers who want to eliminate jargon or identify subtle trends in the language that they choose to use, the clouds provide a visual snapshot of potential trends that are difficult to quantify.  Again, you are absolutely right to say that a decontextualized word cloud isn’t particularly useful.

    I don’t wish away your graphic representation of statistics just because graphics in general have been overused and misused to serve particular ideology.  I advocate for intelligent use.  I hope you do the same for word clouds, which do play an important role in research on language and language use as one analytic tool in an arsenal of many.

  • http://twitter.com/milesparker Miles Parker

    Hi Jacob and all,

    I loved the piece. I’ve been playing with a kind of hybrid approach that might solve some of these issues, and I’d be delighted to have feedback from the news viz folks out there.

    http://milesparker.blogspot.com/2011/12/forecast-spring-clouds-and-sopa.html

    cheers,

    Miles

  • Aknijob

    nice presentation here. from my view, word clouds imply parity of journalistic goals to word frequency. false equivalency rules our era.

  • http://www.facebook.com/profile.php?id=628363553 Kannappan Sirchabesan

    ha haa.. nice finish.. “shame on you” :)

  • http://www.facebook.com/profile.php?id=61001175 Junta Sekimori

    Sure, using word clouds to look for clues in unstructured data like the war logs can be like diving the future in tea leaves, but there are other useful applications for word clouds.

    How about when they are used with structured data like search terms that lead visitors to a website? Wordle handles this kind of data as well via its ‘advanced’ section and really begins to shine when you bring in colour as an additional signifier.

    More on he subject here:
    http://aladata.co.uk/the-silver-lining-of-word-cloud/

  • ebuybag
  • asdfghzxc
  • http://twitter.com/jomynow Joe Antony

    The first thing I did was make a Word Cloud.  

  • http://www.ScottBartell.com/ Scott Bartell

    I never really considered the short coming with word clouds.. I think they’re so popular because they are so easy to make. Anyways, great insight!

  • B_Manx

    where are the phrase clouds?

  • B Law

    I would like to replicate the Tea Party example for a presentation on this topic, do you have the text handy or remember where you found it? Thanks!