Two weeks ago, NRKbeta, the tech vertical of the Norwegian public broadcaster NRK, published an explainer about a proposed new digital surveillance law in the country.

Digital security is a controversial topic, and the conversation around security issues can become heated. But the conversation in the comments of the article was respectful and productive: Commenters shared links to books and other research, asked clarifying questions, and offered constructive feedback.

The team at NRKbeta attributes the civil tenor of its comments to a feature it introduced last month. On some stories, potential commenters are now required to answer three basic multiple-choice questions about the article before they’re allowed to post a comment. (For instance, in the digital surveillance story: “What does DGF stand for?”)

The goal is to ensure that the commenters have actually read the story before they discuss it.

“We thought we should do our part to try and make sure that people are on the same page before they comment. If everyone can agree that this is what the article says, then they have a much better basis for commenting on it.” said NRkbeta journalist Ståle Grut.

Forcing users to take a little extra time to think about the comment they’re about to post also helps them think about tone, NRKbeta editor Marius Arnesen said. “If you spend 15 seconds on it, those are maybe 15 seconds that take the edge off the rant mode when people are commenting,” Arnesen said.

NRKbeta is one of the few sections within NRK that actually has a comment section, and the blog’s dedicated readership has built a community in the comments and typically has pretty positive conversations, Grut and Arnesen said.

However, when NRKbeta stories — such as the story on digital surveillance — are placed on the main NRK homepage, they attract readers who aren’t regulars, which can bring down the level of conversation.

Last summer, for instance, the site posted a story about an online forum that posted photos of underage Norwegian girls. The comments on the story devolved into personal attacks and other questionable behavior.

“We have a pretty good commenting field to begin with, mainly because it’s a niche product,” Arnesen said. “It’s a lot of tech guys, smart people who know how to behave. But when we reach the front page, a lot of people that are not that fluent in the Internet approach us as well.”

Experiences like that inspired NRKbeta to try and find a way to improve the commenting experience.

They settled on the quiz function because they thought it would ensure that commenters had at least read the story and had a common set of facts on which to base the discussion. NRKbeta also thought that the quiz functionality might help keep the discussions on topic.

“We’re trying to establish a common ground for the debate,” Arnesen said. “If you’re going to debate something, it’s important to know what’s in the article and what’s not in the article. [Otherwise], people just rant.”

NRKbeta developer Henrik Lied built the tool as a WordPress plugin, with the questions randomized for each user.

Grut acknowledged that the comment quiz was fairly easy to get around for people with technological experience. One of NRKbeta’s readers even posted a script to show others how to get around the quiz. Grut said the site had to tell them: “Guys, this is not for you. We know it’s easy to modify some code in your inspector…it’s for the people who approach our articles with the intent of just ranting before they even look at the article.”

While many outlets have gotten rid of comments and outsourced reader conversations to platforms such as Facebook, others like NRKbeta are working to improve on-site conversations. Last week, Google parent company Alphabet announced that it was working with The New York Times, The Economist, The Guardian, and Wikipedia to test a new tool called Perspective that uses machine learning to identify “toxic” comments, which it defines as “rude, disrespectful, or unreasonable comment that is likely to make you leave a discussion.”

Similarly, the Times, The Washington Post, the Mozilla Foundation, and the Knight Foundation launched the Coral Project, an initiative to create open-source tools to help news orgs improve their on-site community.

There’s also the startup Civil, which created its own commenting platform that requires users to grade the civility of comments before they’re allowed to post their own.

NRKbeta, meanwhile, is still in the early days of its experimental commenting quiz. The quizzes have only appeared on a handful of stories, though if it proves to be successful, it might be expanded to every story NRKbeta publishes. For now, the reporters who write the stories are writing the quizzes, though NRKbeta might change the workflow if it starts publishing quizzes on every article.

It’s too early to say whether NRK will expand its use of commenting and the quiz tool beyond NRKbeta. But Grut said he believes that NRK should be fostering productive conversations as part of its mandate as a public broadcaster.

“We want to provide a way in for people so that they get more value for their money…they have to pay thousands of kroners every year for us to do our job,” Grut said, referring to the (high) license fee Norwegians pay to support the broadcaster. “We have always been thinking that comments are an important part of what we do.”