Nieman Foundation at Harvard
America’s Test Kitchen, “the Consumer Reports of cooking,” wants to grow to new platforms
ABOUT                    SUBSCRIBE
Aug. 31, 2011, 2:30 p.m.

Mirror, mirror: The New York Times wants to serve you info as you’re brushing your teeth

Meet the R&D Lab’s latest: a proof of concept in the form of a “magic mirror.”

Information is everywhere — in the world, in your home, everywhere. In today’s pair of videos from my visit to The New York Times Co.’s R&D Lab, Brian House, The Times Co.’s Creative Technologist for R&D, demonstrates the Lab’s, er, reflection of that idea — in the form of a data-bearing mirror. The device (working name: “the magic mirror”) uses Microsoft’s Kinect motion-sensing technology to read physical cues from its user; it uses voice recognition technology to detect verbal cues. (In the videos, you’ll hear House talk to the mirror, Snow White-style.) The mirror also uses the the Times’ powerful APIs to serve up information on-demand.

The device, within its notional home, would replace the standard bathroom mirror. And like the R&D Lab’s screen-topped table, it’s all about bringing a new kind of intimacy to the news experience. You can use it, say, to browse Times headlines, or watch Times videos, while you’re brushing your teeth. You can use it to schedule events on your personal calendar, or to shop online, or to exchange messages — from the classic “buy milk” on up — with other members of your household. While the mirror is capable of serving (relatively) traditional forms of content — individual articles, videos, etc. — via its screen functionality, even more striking is its experimentation with information that has, directly, very little to do with the Times itself. In exploring the realms of health and commerce alongside more standard editorial content, the Times Co. is hinting at the products we might see when news organizations expand their scope beyond the news itself.

Essentially, the mirror fuses news — and, in this case, a highly branded, New York Times experience of the news — with all the other forms of data that we encounter in our daily lives. Again, the “information shadow” idea. By building a device that is both a screen and a mirror, the R&D Lab can experiment with the ways to combine the personal and the informational in ways that (it hopes!) aren’t intrusive, but rather helpful and, in that, welcome. This is The New York Times Company acting not just as a curator of information about the wider world, but also as a curator of the information that punctuates, and complicates, and in some sense defines its customers’ personal lives.

Here are the videos’ transcripts.


So one more thing with health. In an off-the-shelf product like Claritin, here, you know, even though it’s not a fancy digital device, it can still participate in information exchanges by virtue of an RFID present in the packaging. So we put it on our shelf here; it’ll bring up directions on how to take it — it’s a lot easier to read than on the box itself.

It can also, by virtue of it knowing what my schedule is — and maybe I have a trip coming up, and it knows that I’m using this product — that creates another information intersection that we can use to query our database, and bring up an article that might be interesting.

And it’s also a retail opportunity. So, say, based on my use of this product, that a local retailer wants to promote something to me.

“Mirror: show coupon.”
“Mirror: show coupon.”

I can call up a coupon here, and then save it to my phone, and then go to a physical location to redeem it. So, again, it’s a conversation that happens in front of the mirror, but then it can drive behavior elsewhere, out in the world.

A similar thing with prescription information. There’s a lot of information behind this, and I can call it up by putting it on the shelf. So it’s personalized to my particular prescription: It shows my doctor and how many days I have left with this prescription, and when my next appointment is, etc. And you can imagine knowing what you’ve taken and what you haven’t taken — or, if you have a family, knowing which medication is yours, or for elderly people to help them take their medications on time. So there’s a lot of possibilities in this domain, too.


So we have another mirror under development, which will have a few more bells and whistles and focus on the retail experience — which is another domain in which an interface like this might make sense for it to play a part. We do a proof of concept here, quickly:

Mirror: show retail.

So this is fun because it’ll find my face — and from my perspective, the tie lines up. It’ll attempt to match the colors to put on the tie. So, again, it’s all about context. And you can see previous outfits that I may have been wearing, and from there make recommendations about what I might like to try on next, etc. We’re working on this, and so we’ll have some more to come.

POSTED     Aug. 31, 2011, 2:30 p.m.
PART OF A SERIES     The New York Times R&D Lab
Show comments  
Show tags
Join the 15,000 who get the freshest future-of-journalism news in our daily email.
America’s Test Kitchen, “the Consumer Reports of cooking,” wants to grow to new platforms
“We’d like to move to other platforms, particularly as we see the changes in how people consume television.”
A program from Poynter and ONA is helping foster a community of female leaders in digital media
The Women’s Leadership Academy provides camaraderie and concrete advice beyond a bundle of platitudes.
Come talk ad blockers with Nieman Lab and a set of experts in New York
We’re having our first event in New York City with industry leaders: Wednesday, December 2 at 6 p.m.
What to read next
How one blog helped spark The New York Times’ digital evolution
“I certainly had editors tell me that I shouldn’t be wasting my time on Bird Week. But that was the best part of City Room…We were like unsupervised children.”
572News outlets left and right (and up, down, and center) are embracing virtual reality technology
Among those experimenting is The Wall Street Journal, which plans to open source its 360-degree mobile video and VR technology and hopes to turn VR into more of a mainstay of its storytelling.
502Podcasting in 2015 feels a lot like blogging circa 2004: exciting, evolving, and trouble for incumbents
The same trends we saw a decade ago — professionalization on one hand, platformization on the other — sure seem to be playing out again.
These stories are our most popular on Twitter over the past 30 days.
See all our most recent pieces ➚
Fuego is our heat-seeking Twitter bot, tracking the links the future-of-journalism crowd is talking about most on Twitter.
Here are a few of the top links Fuego’s currently watching.   Get the full Fuego ➚
Encyclo is our encyclopedia of the future of news, chronicling the key players in journalism’s evolution.
Here are a few of the entries you’ll find in Encyclo.   Get the full Encyclo ➚
Hechinger Report
BBC News
The Christian Science Monitor
The Batavian