Tough love: Gawker finds making it harder for comments to be seen leads to more (and better) comments

Explore Harvard's Nieman network Nieman Fellowships Nieman Lab Nieman Reports Nieman Storyboard

Fast Company is hacking the newsroom. Here's why: nie.mn/14IyYjb

Pushing to the future of journalism — A project of the Nieman Foundation at Harvard

April 13, 2010, 1:22 p.m.

Tough love: Gawker finds making it harder for comments to be seen leads to more (and better) comments

By Joshua Benton

That chart is, for news organizations seeking to tame their commenters, perhaps the best evidence yet that adding a few obstacles for those seeking the leave their mark on a web page can actually lead to more comments. And better ones, too.

That chart (bigger version here) tracks the number of comments left by month on the Gawker Media blog empire, Nick Denton‘s collection of themed sites (Gawker, Gizmodo, Deadspin, et al.). It covers September 2005 to the present. See that big dip on the right? That’s when Gawker implemented a new, stricter commenting system, in which trusted commenters get preferred access to readers and the unknown hoi polloi have to audition for an audience. (We wrote about it at the time; in an internal memo, Denton wrote about “taking back the site from some commenters who thought they were in charge” and said “we’ll be able to encourage the kind of discussion that *we* want — not one that is dominated merely by the most prolific of our commenters. It’s our party; we get to decide who comes.”)

In essence, Gawker’s “class system” means unknown commenters get stuck behind a “show all discussions” link few users will click. What most readers will see are only the musings of trusted commenters and the few comments from the riff-raff that either Gawker staff or trusted commenters have decided to promote — the “featured discussions.” (The system also put the most recent comments on top, not on bottom as at most sites. That would seem to reduce the possibility that a dumb early comment would sway the chain of comments that follow it into irrelevance.)

As the chart shows, the shift led to an immediate decline in comment volume. (Interestingly, the biggest drop seems to have been at Jezebel, Gawker’s women-centric site. Attention communications and/or gender studies grad students: There’s a thesis somewhere in there!) But comments quickly rebounded and have since skyrocketed at a much faster slope than before the switch. Some of that is no doubt related to Gawker’s overall increase in traffic, but the scale of the increase is still remarkable.

Gawker Media CTO Tom Plunkett posted the chart on his blog. His interpretation?

Quality *and* growth — it’s possible! We launched tiered commenting mid-year 2009, and introduced a new process to manage comment volume. Note the dramatic drop in volume, and the subsequent rise (double in 9 months). With this increase, Gawker still has the best commenting system/experience out there — and I usually hear the same from people that want to share their opinion…

Though there were some calls to do so, purging commenter accounts is not a solution for the out-of-control commenter community. Nor is a large moderation staff. We believe pruning, and a commenting platform as we have implemented, will lead to increased participation, while at the same time encouraging quality. This data, and the subjective opinion of many, seem to back this assertion.

I’m a regular Gizmodo and Gawker reader (and less regular Lifehacker and Deadspin reader), and I can add to the subjective opinion that average comment quality is higher than before. But “better” isn’t the only scale on which you can measure comments. I think the audition-for-an-audience nature of the new system also makes the comments quippier; Gawker comments can feel like a bunch of wannabe Henny Youngmans spouting one-liners and seeking attention. But that vibe may have more to do with Gawker’s content and tone than the details of its commenting policies.

In any event, complaining about awful commenters seems to be the first thing any gaggle of journalists does when lamenting the new news reality. The default solution has been to say every commenter should have to use his or her real name — a solution with practical as well as ethical problems. (Although Facebook Connect may be taking away some of the practical concerns.) Still, there’s a whole world of ways a news site can improve the tenor of its comments while keeping itself reasonably open. Gawker Media’s success is one example of how.

What to read next

Caroline O'Donovan June 12, 2013

Would you click a “Respect” button more than a “Like” button? Experiments in tweaking news reader behavior for democracy

The Engaging News Project want to know if that and other small cues and prompts can encourage people to seek points of view different from their own.

Dahlia

A couple of comments on the Gawker commenting system with regard to when the announcement about overhaul was made on the site:

1) It was stated during the overhaul that by and large, people would be losing their stars. This happened for about five minutes before all starred commenters got theirs back.

2) By and large, “gray” comments are read by starred commenters, whether they are promoting them or not. I’ve seen nothing to suggest that they’re being ignored in the slightest, and I would guess that an enormously large percentage of comments have “Show all comments” as a default.

3) The number of stars given out in the last couple of months seems to have shot up, which I’d imagine is largely responsible for a sharp increase in comments as well.

Yup, I spend way too much time there.

~DahlELama, starred commenter since some weekend last summer when I discovered I had one on Monday, but then had it taken away right after the announcement was made for not liking Jessica Biel, but then had it returned shortly thereafter.
http://jonathanstray.com Jonathan Stray

Take note: banning anonymity ain’t the answer. Reputation-based filtering is. This and more was understood by slashdot in 1999, see here.
Chris Clonts

This leaves out a key component: Comment VIEWS. Did those increase as well? Becuase they make up the bulk of comment traffic. So if overall views went down, but comments went up, it’s either a lose situation or a wash.
http://www.niemanlab.org/ Joshua Benton

Chris: Gawker puts comments on the same page as the post, so I don’t think this would have a serious down impact on pageviews. And in any event, as the links in the post say, traffic is substantially up over this period; whether or not you want to attribute that to comment strategy is another question. They’ve got enough scale that the difference is more between having 30 comments and 70 comments on a post, so I wouldn’t expect the difference to be huge.
http://thomped.com Tom Plunkett

@Jonathan Stray: Excellent point! Others (Slashdot in particular) have been down this road — publishers should take note.
Pingback: Improving Reader Comments « Ethix2.0
Pingback: Slant, Tilt and Bias - Steve McNally - Musings of a New News Startup - True/Slant
Pingback: Slant, Tilt and Bias - Steve McNally - Musings of a New News Startup - True/Slant
http://joeclark.org/weblogs/ Joe Clark

Gawker sites aren’t newspaper sites and their solution, like Israeli airport security, doesn’t scale. Comments on factual newspaper articles, a decade of empiricism proves, do not work. Possibly they could work on editorials, with massive moderation.
http://wiredpen.com/ Kathy

Hi, Joshua — I’d like to see more data on the change in traffic over the same time period.

The large chart suggests that comments doubled over a 10 month period. What about combined site traffic? How does this compare with prior periods — how fast was commenting growing compared to page views? How uneven is the growth (traffic/site compared to comments/site)?

As someone else pointed out, Slashdot figured this out a decade ago.
http://www.niemanlab.org/ Joshua Benton

Hi Kathy: You can see pageview data for all of Gawker Media’s sites here:

http://nickdenton.org/5322904/weekly-traffic-on-gawker-sites

As you can see, the slopes are all generally in the upward direction, but the growth in pageviews from early fall 2009 to today isn’t anywhere near the more-than-doubling in comments over the same span.

And let me just say that I’m not convinced of the effectiveness of Slashdot-style moderating on sites that are not Slashdot. Works great for them, sure, but I sincerely doubt that a newspaper site’s readers would find the endless fiddly-ness of upmodding and downmodding and karma and thresholds and “Flamebait vs. Troll” as pleasing as a bunch of Slashdot nerds do. (And I say that as a nerd!)
Fred

I am not sure this is the right approach. Personally, I commented several times on Gawker and my comments were never promoted or shown, as far as I can tell. I feel this was a bit of a putdown – I spent may be 20 minutes writing one of those comments and the fact that it remained essentially unapproved made me just a little bitter – at the time I was let to believe that any worthy/interesting comments will be shown. So, obviously I stopped wasting my time.

This “elitist” approach where a group of insiders makes “snarky” one-liners that are somehow considered more illuminating and worth publishing than what any outsider may possibly offer results in a distinct feeling that you’re not participating but over-hearing a private conversation. It’s as if the site were an exclusive club and you’re being kept outside at the velvet rope.

Sorry, but as much as I find Nick Denton a clever dude and all, I am not willing to endure the humiliation of this hazing process. You just go ahead and have fun inside, guys, but I’ll take my 2 unworthy eyeballs and 10 unworthy fingers somewhere else.
Pingback: Kotaku Off Topic: Joss | Kotaku Australia
Luis M

Well my issue is one with uniformity and consistency.

The Gawker site I frequent the most is the electronic gaming site, Kotaku, so many of the comments there are silly by nature. The star users there seems to run the gamut from “Thoughtful, intelligent post” down to “Snarky one liner”, with a considerable amount of people being promoted in the latter end of the spectrum. What bothers me the most is just how lopsided the trend seems to be across all Gawker sites that the “one-liners” tend to be the least intellectually stressing comments yet the most often promoted to star status.

This tends to create an environment of posters who are desperately trying to create an identity for themselves based on repeated use of some quirk or posting style in order to gain some recognition. This is the most pronounced in the more fiction oriented sites.

In shorter terms, it is slowly growing into something more memetic. Something that could remain a problem later on down the road.

Usually this will not work for the average poster, and their effort will be in vain. I have noticed that the amount of persistent users have remained the same, whereas I see a great deal more of “one-time” users off to set their stakes for the “starred elite” only to return broken when they fail to reach that milestone.

This could be solved with a reputation based filtering, but the results shown by these statistics means it’s better to just maintain what’s currently in place now and just tweak with it. Mending is better than ending…
Pingback: links for 2010-04-14 » Wha'Happened?
Pingback: Andrew Golis » Blog Archive » links for 2010-04-14
Pingback: Stop Press for April 14th | booktwo.org
Pingback: Journalism Today - Tracking Media Changes
Pingback: Gawker: Weniger sichtbare Kommentare führen zu mehr und besseren Kommentaren
sage2127

It is interesting to see Gawker, et al evaluated on the basis of the quality of its comments/commentators. What this means, apparently, is that it is now acceptable to profit from printing rumor and innuendo which can devastate a person’s reputation or career, and that the Nieman “Journalism” Lab is acknowledging what Denton does as a legitimate form of reporting. While I am not disputing the existence and power of blogging, this is an empire devoted to malicious, even joyful character assassination. Why is it being treated here as something that is on a par with “legitimate” news outlets?
Pingback: By giving commenters trust, Gawker did the right thing – Dutch Problogger
Pingback: This Week in Review: News talk and tips at ASNE, iPad’s ‘walled garden,’ and news execs look for revenue » Nieman Journalism Lab
Sal

Kathy was on the right track by requesting more data. Although the subjective quality of user comments and the number of comments have both increased since their system change, you need to realize that the perceived uptick in quality is based on those comments that have been “promoted” and easily viewed. There is nowhere near enough basis to conclude that the TOTAL number of quality comments/commenters has increased.

As you mentioned, very many comments – by the “unapproved” masses – are hidden behind an extra click. Many more comments are replies, which are also initially hidden by the Gawker system. This leads to a vast number of redundant comments & replies as can be easily be verified by making that extra click. I’ve seen enough authors of approved or promoted comments actually edit their comment to discourage further answers/replies because they’re all the same. This is not a picture of a system that works as well as it’s being advertised.

Since I am a regular reader of some of the Gawker sites, including unapproved comments, I’d actually argue that the system works well to highlight quality posts (assuming you like snarky one-liners) or longtime contributors, but increases the TOTAL number of comments only by causing a huge number of redundant, and therefore non-quality posts.
Pingback: links for 2010-04-17
Pingback: Success in Vegas and NPR in coffee shops - James Cridland
Pingback: Läsvärt – vecka 15 2010 | bryggare.nu
Pingback: Pigsaw Blog » Blog Archive » Bookmarks for 19 Apr 2010
John Doe

Joshua Benton or Anybody:

I am sure the Gawker family have a core user they target for ad purposes. My question, has the new commenting system improved that core audience whether they spend more time on the page, or clicking on ads, or attracting more users of that type, etc?

John Doe
Pingback: Gizmodo Made the ‘Next iPhone’ a Great Story | Joe Wilcox
Pingback: Trust, But Verify: TidBITS Commenting System Succeeds « MacPadd.com
Pingback: An answer for online anonymity « Engage: McNeely Pigott & Fox Public Relations
Pingback: A case for better comments «
Pingback: How should news sites deal with comments? | Spark | CBC Radio
Pingback: This Week in Review: News talk and tips at ASNE, iPad’s ‘walled garden,’ and news execs look for revenue | Mark Coddington
Pingback: OutofNets » Blog Archive » dd
Pingback: Internet Strategy for News Organisations » Session 4: Social Software
Pingback: Rolling Stone’s late start on McChrystal costs it comments » Nieman Journalism Lab
Rob

The majority of the Gawker’s trusted commentators are overwhelming leftist or liberal, depending on their POVs. Either they won’t promote comments that are of conservative leaning or allow their fellow leftist/liberal commentators to attack conservative or right-leaning commentators for all to see. It’s almost the same as Big (Breitbart) sites, only in reverse but surprisingly, conservative/right-leaning commentators are more tolerant of leftist/liberal commentators there. Gawker’s “trusted” commentators are more intolerant of anything that doesn’t conform or reinforce their POVs.

I prefer the Big’s IntenseDebate commenting system over the Gawker commenting system because ID system allows people to thumb up or down comments. No need to hide disagreeable comments, except for inappropriate or offensive comments (which get deleted). And disagreeable comments invite spirited debates at the Big sites. Not so much at the Gawker sites.
Pingback: More, better website comments? « Jason Kandel
Pingback: Community Engagement, Best Practices: How to make your comments section feel more like Cheers and less like a bar fight « Sarah Fidelibus
Pingback: More growth for Gawker comments, and more power to elite commenters » Nieman Journalism Lab
Pingback: Evolution of comments needed
Pingback: Catalysts: The Globe and Mail’s community brain trust » Nieman Journalism Lab
Pingback: Gawker Media’s redesign: What’s it mean for the future of news sites? Pt. 1 « Alex Reports
Trabajo medio tiempo

This is a very interesting point of view.

Trabajo de medio tiempo
Pingback: By giving commenters trust, Gawker did the right thing | Ernst-Jan Pfauth
Anonymous

Gawker wants readers to follow them on Twitter or Facebook. But they’ve failed my audition.
http://myindigolives.wordpress.com/ Ellie K

Hello @jbenton:disqus ! My apologies for asking so long after the fact, but might you re-post that link to the page view data for Gawker Media sites, referenced in your comment? Nick Denton dot org no longer exists: It now redirects to a fancy Facebook page.

I found this article while surreptitiously browsing the bookmark libraries of CiteULike users (academic citation-type social bookmarking site) of all places! I’ve never seen anything other than peer-reviewed publications there, so it was kind of a surprise. I am a mild fan of Gawker… no, an enthusiastic fan. I enjoyed reading your post, very much! It was thought provoking, given my fascination with spam and detection methods as both a web-dweller and a statistician.

I’m asking you to do something for me, so I’ll offer something (minimal as it is, but best I can do for now) in return. As a Gawker comments reader, you may find their recently overhauled approach of interest. I did! Here’s a sample: http://gawker.com/5856588/this-week-in-commenter-executions-who-is-it-gonna-be from Nov. 2011. Thanks for considering my request.
http://www.niemanlab.org/ Joshua Benton

Hey Ellie: You can see author stats and other useful numbers on each Gawker Media site. All you have to do is add /stats to the end of the URL. So gawker.com/stats, gizmodo.com/stats, deadspin.com/stats, etc.

For the overall traffic numbers for the site, follow the Quantcast link at the top of one of those pages.
Anonymous

Yea gawkers commenting system is just even more broken now. When you setup barriers like that you only get the die hards or no lifers to comment, who in their right mind would waste their time “auditioning” with invisible comments trying to curry favor until someone agrees with you enough to promote you into visibility. It is like some sick one state party system where you have to show your ideological credentials and loyalty before you are accepted as a fellow comrade. Essentially what gawker has done is turn the comment system into one policed by a sick little clique of no lifers. They will enforce opinion and it does show, as for the female centric sites, I’m sure you will see very little disagreement anymore in the comment section, which is strange because the feminist gawker type sites are actually link bait tabloid feminism which take every opportunity to take the most outrageous stance on anything possible to generate traffic. Just that now they will have to generate that traffic without any honest responses to their posts..

As fred below has said, this system weeds out people who just won’t waste their time on such nonsense, so the idea that it filters for quality posts is a nonsense, it filters out a whole lot of quality posts from the first pass.