Twitter  What’s new in digital and social media academic research this month nie.mn/1mZ8L5w  
Nieman Journalism Lab
Pushing to the future of journalism — A project of the Nieman Foundation at Harvard
algorithm-city-cc

There’s no such thing as an objective filter: Why designing algorithms that tell us the news is hard

Technologists and humanists take different approaches — and speak different languages.
Email

We are all immersed in an incomprehensible abundance of available information, and we can only read or watch or consume some meaningless fraction of it. What we see, and what we don’t see, is heavily mediated by information filtering algorithms: Google web search, the Facebook news feed, personalization and recommendation engines of all kinds. Filtering algorithms shape our knowledge and our society. They are now a permanent part of what it means to perceive the world.

So it’s important to design them well, to make good choices here. What makes a “good” filtering algorithm? There’s no easy answer.

How can there be? The question of who should see what when is not like the question of how far the moon is from the Earth, or what the mayor ate for breakfast today, or whether Elvis is still alive. Those questions have simple answers that are either right or wrong. But choosing who sees what information is not like that at all: There’s no one answer, just many possible visions of the type of world we’d like to live in.

It’s not just about code. All filtering algorithms operate on human input.

Yet we still need answers. Otherwise there’s no way to build the algorithms we must have, and no way to critique them.

Different disciplines have different approaches to this problem. At the risk of caricature, let’s say there are two broad camps here. The “technologists” are engineers, computer scientists, people with training in quantitative fields. They are the people likely to be directly responsible for building our filtering systems. The “humanists” are editors, curators, writers, sociologists, humanities scholars. They spend their life on the handcrafted work of deciding that this, and not that, deserves our attention — or examining the consequences of such choices. These two cultures need each other, but they don’t seem to speak the same language.

Technologists have long used two numbers to measure how well a search engine works. “Precision” counts how many of the returned items were actually relevant to a user’s query, while “recall” measures how many relevant items the search should have returned, but didn’t. Together, these two percentages — false positives and false negatives — give a clear indication of search engine performance. You can make a change to the algorithm and see if the numbers go up or down, and by how much. Recommendation engines are another type of filtering system, and in practice each optimizes for some numerical value. A retailer like Amazon might measure how much each customer ends up buying, while Netflix is more concerned with choosing movies that you’re going to like (as measured by the ratings you assign). News personalization systems often use the number of clicks on your customized headlines as a proxy for success.

These sorts of measures are crucial for engineering work, but of course they miss much. Search engine performance metrics only work if you, the human, already have a clear idea of what counts as “relevant” for a given query. Dollars and clicks are useful metrics, but they squeeze rich social interactions into a limited economic mindset.

Humanists tend to think about things like whether the user “needs” the information, or whether it might challenge them, inspire them, or teach them something new. Or they might be concerned about filter bubbles, the idea that personalized information filters will end up telling us only what we want to hear, fragmenting us into smaller and smaller factions that never really talk to one another. Other people wonder about serendipity in online information systems, imagining ways to help us discover things we might never have looked for. All of these people ask difficult questions about how our media shape our societies, and what’s good or bad about that.

But how can we actually know if any particular system is producing “filter bubbles” or not? And given the choice between two algorithms that might be quite similar, how should we pick one over the other? Descriptions of humanistic qualities are usually far too vague to be turned into code. Public Insight Network co-founder Andrew Haeg has suggested that a good filtering algorithm should align with Maslow’s hierarchy of human needs. Sounds great, except that I have no idea how to translate that into an algorithm, or how I would test my code against the real world to see if it satisfied those needs. As sociologist Stuart Hall famously put it:

News values are one of the most opaque structures of meaning in modern society. All “true journalists” are supposed to possess it; few can or are willing to identify and define it. Journalists speak of “the news” as if events selected themselves…Yet of the millions of events which occur every day in the world, only a tiny portion ever become visible as “potential news stories.”

Technologists often have a refreshing pragmatism here: Facebook ran a controlled experiment with 250 million users to see what effect removing something from your news feed has on whether you or not you eventually see it. Google continually tests search algorithm changes by asking thousands of users whether they like the results in Column A or Column B better. But this kind of engineering is no substitute for envisioning truly new ways of organizing information; decades ago, Horst Rittel and Melvin Webber explained why “optimization” will not solve societal problems, inventing the term “wicked problem” in the process.

Conversely, humanists are often unwilling or unable to grapple with the dirty technical details of how algorithms are built, and what is actually possible. It took a computer programmer, Seth Finkelstein, to explain how algorithmic constraints create the sociological effects of Google’s web search. Nice work, and there’s an argument to be made that anyone contemplating filtering algorithms should understand the engineering involved. But we’re also going to need simple ways to explain all of this to non-specialists. Many people accused Twitter of censorship when #occupywallstreet failed to make the trending topics list despite simultaneous protests in dozens of cities. In fact there was no censorship, just a quirk of the trend-finding algorithm — an algorithm that “claims to know the mind of the public.”

Moreover, it’s not just about code. All filtering algorithms operate on human input, and some are much more reliant on humans than others. Web search engines look at the links between pages, links that were put there by people. Collaborative filtering algorithms based on “likes” or voting (à la Reddit, Digg, Slashdot, etc.) solicit direct feedback. The result of any filtering algorithm depends on a complex interaction between the code that implements it and the people who use it. The same algorithm in different social contexts, or administered by different people, can produce wildly different results. We have to look at culture and code together.

So filtering algorithm design is one of those wildly interdisciplinary problems. The challenge is to imagine systems that:

  • forward societal goals that we think are important, yet are precise enough to be phrased as performance yardsticks,
  • combine algorithms with humans in a productive way, and
  • can actually be built with available technology.

That’s very hard. It requires a rare type of cross-domain thinking, because we don’t yet really know how to combine the pragmatic demands of technology with the social aspirations of the humanities. But it’s also an exciting time to be working in digital journalism, where these two cultures meet every day.

Algorithmic image by Anders Hoff used under a Creative Commons license.

                                   
What to read next
how-why-explainer-explanatory-cc
Ken Doctor    
When people talk about explanatory journalism, the focus is on new players like Vox and FiveThirtyEight, or on giants like the Times and the Post. But can connecting the dots trickle down to the local level?
  • TerribleTerry

    A person “arrives” at an article via any number of
    ways.  Perhaps they found the link at the host website’s “front door.” 
    Perhaps they got it through a Google Search.  Or from a blog.  Or from
    an e-mail.  Or from an RSS feed display.

    The reason the visitor chose the article had at least something to do
    with their personal interest in the article’s content, regardless of how they
    located it.

    If – and it’s a BIG if – the chosen article were categorized by its
    underlying theme(s), that theme could allow the publisher to forge links
    to other articles representing that same theme.  Thus, for the second
    and beyond article, the navigation (filtering?) would be the match
    between the viewer’s personal interest, and the accuracy of the theme
    categorization.  If this were done properly, the visitor need not know
    how to describe that theme, but could use the embedded links to do the
    “filtering” for them.

    The newspaper that figures out how to implement such a theme-based
    categorization for its articles, will be the (rare) one that moves toward a
    business model that succeeds.
     

  • http://www.luxundpartner.at/ Dominik

    So we’ve to check and optimize our personal filters

    And more than that we’ve to protect ourselfs against unhealthy news

    I don’t want to eat addled food too

  • TSteichen

    Once a visitor has found (by whatever means) an article that interests them, it makes enormous sense for the host news site to help that visitor more easily find other articles that interest them. 

  • Lisa Ray

    It took me an hour to understand the algorithm but it was really interesting learning about this. Thanks so much..now i understand what facebook and twitter depend on user input..Sample Resume Objectives