When you read stories in The New York Times’ app, does you count that as reading a story online? What about news from Twitter? Is it only “online” news if you’re reading it from a computer? And, come to think of it, are the New York Times stories you read digitally really “online news” at all, or are they newspaper stories?

These are the kinds of questions that Pew grapples with when it formulates its surveys about Americans’ online news reading habits. In a Medium post, Pew research associate Elisa Shearer laid out some of the ways that the organization has changed the wording of its questions over the past 20 years (it started asking about online news in the mid-1990s).

“The internet is a relatively new source of news for Americans, compared to, say, TV or radio,” she writes. “As a result, it can be hard to design surveys that get at the uniquely complicated nature of online news.”

Back in 1996, our surveys asked Americans whether they got news through "computer on-line sources." (Cue the AOL dial-up sound.) Lots of things have changed since then, including the way surveys ask about people's online news habits. New Decoded post: https://t.co/5hLLpOmc3Q pic.twitter.com/1NmoanhTxk

— John Gramlich (@johngramlich) July 1, 2019

For instance:

In a survey after the 2006 midterm election, we tried to categorize all the forms of online news into four groups: newspaper websites, TV news websites, “Internet news sites such as Google news or Yahoo news,” and “other.” In that survey, the “other” category got as many responses as any of the other three categories, which is not something you usually want to see in an “other” response.

The lesson was clear: There was a large section of online news sources that didn’t fit neatly into the buckets we had initially tried to use.

She also notes that part of Pew’s job is to make sure that respondents clearly understand what’s being asked. Back in 2012, that meant using asking people if they’d ever downloaded “an application or ‘app’ that allows you to access news on a cell phone, tablet or other handheld mobile device.” Today, nobody needs the word “app” defined for them. But it’s still important to try to distinguish the ways that people are thinking about where they’re getting their news. In 2019, Pew is solving this by asking about people’s online news consumption in two different buckets:

First, we combine the news people get on websites, apps, or social media sites — focusing on the platforms where people are getting their news. Second, we ask about the physical devices where people get news online — on desktops, laptops and mobile devices.

This, of course, is also subject — and likely — to change.