Sometimes you just get lucky with the editorial calendar.
About a month ago, Superhuman CEO Shishir Mehrotra agreed to be a guest on Decoder, the podcast from The Verge hosted by editor-in-chief Nilay Patel. Superhuman is the company formerly known as Grammarly, which is now just one of its AI-focused productivity tools, and Mehrotra and Patel would have already had plenty to talk about.
But then journalists started noticing a little-known Grammarly feature called “Expert Review,” which offered users suggested improvements to whatever they’re writing. But the proposed changes were framed as coming from real writing experts — journalists, novelists, academics, and more. It wasn’t AI suggesting a plot twist — it was Stephen King. It wasn’t AI proposing a cleaner way to explain science — it was Carl Sagan. And it wasn’t AI pointing out a way to improve your tech column — it was Nilay Patel.Writers, as one might expect, were not thrilled to see their names being used to lend credibility to AI editing suggestions. (Even me!) Technology journalist Julia Angwin filed a class-action lawsuit against Grammarly, seeking damages. By that point, Mehrotra had announced the demise of “Expert Review” after eight months, saying he wanted “to apologize and acknowledge that we’ll rethink our approach going forward.”
Give Mehrotra credit for showing up for his interview with Patel anyway, which was released this morning. After a brief chat about Superhuman’s product strategy overall, Patel dove straight into the dispute:
Patel: You do not have our permission to use our names to do this. You had little check marks next to the name that indicated it was somehow official. People did not like this, I did not like this, and you removed the feature. Tell me about the decision to launch this feature with names you didn’t have permission for and the decision to unlaunch the feature.
Mehrotra apologizes again (“It deeply pained me to feel that we under-delivered”), notes that Expert Review wasn’t especially popular, and that “the feature was not a good feature. It wasn’t good for experts, it wasn’t good for users.”
Mehrotra: For some of [Grammarly’s users], the people they want feedback from are the people they admire. It’s the experts in the world, it’s the people that they’re trying to look up to and trying to model. They try to do that today with LLMs. They go to ChatGPT and Claude and say, “What would Nilay think about my writing?” That was the inspiration for what the user was trying to do.On the other side was what the experts were trying to do. As we formed our strategy here, turning Grammarly into a platform, the first people I called when thinking about this were a set of experts. I talked to some prominent YouTubers, I talked to a really prominent book author, and they all told me the same thing. It’s a really hard world for experts out there right now. It’s really hard to drive connection. If you’re a book author, your path to getting to your fans is you just keep publishing more and more books. And they all heard what we were doing and said, “Boy, it’d be really amazing to develop an ongoing connection with my fans. What happens when they put my book down? Can I still be with them and help them along the way?” It feels like the world shifted against them, AI Overviews stealing a bunch of their traffic and so on. This seems like a much better way to go after it.
That was the inspiration behind it. The team and the feature didn’t deliver. It didn’t deliver on either side of it, really. We ended up with an experience that was pretty suboptimal for the user and obviously suboptimal to the expert. The fundamental reason is something you said last week, that it’s really hard to distill what you would do as an editor based on the outcome of your published work. It’s really hard for AI to do that. We need your engagement for that to be a good feature.
To which Patel had a wonderfully direct response:
Patel: Sure. How much do you think you should pay me to use my name?
Mehrotra’s response is, in effect, attribution and a link.
Mehrotra: It’s really important to think about attribution and think about impersonation, and so on. As an expert, you have a trade you make on the internet. The idea is that when you put content out there, myself included, you hope people use it. You want to refer to other people’s content. You want people to link to you. You really, really hope they attribute you when they do. When somebody uses your content, should they attribute you? Of course. And to attribute you, you have to use your name.There’s a different line which is, should people be able to impersonate you? And I think that is a very different standard. And we saw the lawsuit. Respectfully, we believe the claims are without merit. The idea that the feature is impersonation is quite a big stretch. Every mention was very clearly, “This is inspired not only by this person, but also inspired by a specific work from this specific person, with a clear attributed link to get back to them.”
It’s true that citing the names of specific people can be a powerful way of shaping an LLM’s response. Plenty of prompts start by telling Claude something like, “You are ________, the world’s leading expert in ___________.” You could ask ChatGPT: “Rewrite this essay in a spare, understated style using clear, direct language and short sentences, favoring simple vocabulary over ornate phrasing, emphasizing concrete details and action over explanation, avoiding abstraction and sentimentality, and conveying emotional depth indirectly through precise, unadorned prose.” Or you could just type “Rewrite this like Ernest Hemingway” and save some keystrokes.1
But does that LLM’s response mean the Hemingway estate is now owed money? And does Grammarly saying “This suggestion is inspired by Joshua Benton’s ‘Nieman Journalism Lab analyses'” mean Mehrotra owes me a steak dinner?
Check out the full Decoder episode to hear their vigorous back-and-forth over how, as Patel puts it, “people don’t understand the difference between copyrights and trademarks and names and likeness,” and that “AI is collapsing those differences faster than ever before.”
- And yes, I did get that first version by asking ChatGPT: “I want you to rewrite this prompt — ‘Rewrite this essay like Ernest Hemingway’ — without mentioning Hemingway at all. In other words, make a prompt that requests all of the qualities of a Hemingway rewrite without referencing him specifically.” [↩]
