Minnesota Public Radio‘s Linda Fantin and the Sunlight Foundation‘s Ellen Miller were the stars at an MIT panel a few days ago; I wrote up their discussion. But after the panel, I sat the two of them down to talk a little more about the challenges of running experiments with community-generated journalism. A few highlights:

— Miller: “I think the ability to fail is absolutely part of the culture in which we live. And so, someone will try lots of things, which you know sometimes just don’t work…there’s not much cost to the experimentation.”

— Fantin: “[I]t’s tough to let go of things that haven’t quite reached their potential, but you have to, because there’s so much more coming down the line.”

— Miller: “I think there are all kinds of questions about a community: How do we nurture a community? How do we let them do their own work? Is there top-down control about what they do? What degree? How much can we let go and still have it operating in a single campaign framework? And we all figure these things out as we go along, and no doubt we’ll make mistakes out of it.”

— Fantin: “Talk to anyone who started a citizen journalism site or community. They’ll say, ‘Okay, it took over my life, then it took over my wife’s life, and now we actually have to make money and put food on the table, so we sort of tried to get interns, but we can’t sustain it.'”

Above’s a video of their discussion, with a transcript below. (The video’s soundtrack, if you’re wondering, is an apparently epic game of ping-pong taking place in a nearby rec room.)

Ellen Miller: I think the ability to fail is absolutely part of the culture in which we live. And so, someone will try lots of things, which you know sometimes just don’t work — but because we don’t know how people want to engage with, you know, either fairly wonky information about legislation or critical information, if we don’t build it, we never give people the opportunity to test it. And some of the things have worked far beyond — much better than — what we expected. And some of the websites just weren’t popular, and we couldn’t quite figure out why, and we said, “Oh, they weren’t popular, let’s just take it down.” So there’s not much cost to the experimentation. But partly I think it’s because you have sort of a new and largely successful of the project because Sunlight, you know, is an institution without any legacies. It’s just — it’s really built into the DNA. But it’s something major other institutions, you know, have to work on. Now you can’t really build it into the DNA of reporting a story: Failing, getting the facts wrong, telling the story that’s wrong. But there are certainly elements in terms of engaging citizens, in getting them to tell the stories that work. So, I mean, if it doesn’t work this time, you know, you might try it again or — or not. But we’re beginning to learn.

So one of our examples was — it was successful, but it was a failure in the end. We did a series of distributed research projects in the early days. We do one investigating members of Congress’ spouses, and whether they were employed by their campaigns. And then we did another one on getting people to contribute to a database on earmark requests when they started posting them. And then we realized that if people who worked on Project A, we had no idea if they’ve been secretly working on project B, or who worked on Project C. We said, “Wow, let’s stop that.” We created one platform, Transparency Corps, so that anybody who worked on A or B or C had the opportunity to see what was D, E, and F coming down the road, to begin to build more of a community. Because if you’re interested in these kinds of distributive projects, you’ll be interested in, you know, any number of them, and you get deeper engagement in them. So it worked in the individual pieces, but we knew we were losing these people because we didn’t know quite how to reach out to them again. So I think it’s that experimentation or constantly, constantly iterating on something that worked or that didn’t work until you find things that work.

Linda Fantin: And being able to let go of things that aren’t working as well as they could be and not consider it as a failure.

Miller: Oh yeah, that’s hard. What do you mean? That project is really important. How do we let it go?

Fantin: Right. Right.

Miller: One of the things we discovered partly because Sunlight was so innovative in the early days, we would describe — we would try something and say, “Wow, that’s a cool idea. Let’s do it.” We’d throw it up on the wall and we’d develop it and it would be successful. And then we got another cool idea and then we would do that. And then all of a sudden we realized that we had all these projects. We’d be, “how do we sustain them?” So if you have the image of things sliding down the wall, you know, we’d pick up one and then we became — we realized we had to not just constantly develop new things, that we had to iterate on the things that were successful.

Fantin: Well, absolutely. I mean, I know that I mentioned before we created Budget Hero and launched that in May of 2008, we had no idea that the economy would fall apart and that there would be a $787 billion bailout, and then a stimulus packet, and then suddenly the federal deficit would just bloom, and that there would be new Congressional Budget Office baselines every three months that were significantly different than the months before. And I think probably seven, maybe eight times, we’ve had to do major updates to the game. And that wasn’t something we’d planned on in the financial planning that probably created Budget Hero. And even now, part of it is that in some ways it was a game before its time, because now it’s more important than it ever was before. But, you know, having the funds and the ability to say, “Oh, well, we’re going through all the significant — invest in it yet again” is a big decision. I mean, carrying some of these projects forward, you know — it’s tough to let go of things that haven’t quite reached their potential, but you have to, because there’s so much more coming down the line.

Miller: Yeah, and I think we certainly underestimated, you know — we would always figure out what was the cost to build something, but then to —

Fantin: To maintain it?

Miller: So that’s something we’ve certainly learned. So that’s now all built in to, you know —

Fantin: It’s one of the first questions you asked, which is great: Who’s gonna own this, and who’s gonna do it, and when are we gonna shut it down?

Miller: And that’s why I asked you the question, like: How many people does it take — you built this community. How many people does it take to maintain it and to really use it, in a popular way?

Fantin: It’s a good question because —

Miller: Because most groups don’t think about that thing — about community, and you know, it’s a little like magic, which is: “The community will just thrive.” No, you have to nurture this community. You have to add to it. You have to engage with them.

Fantin: Absolutely.

Miller: There has to be a person or a team of people who work with them, and I don’t think people realize that. There was this idealistic vision of community journalists, right?

Fantin: Talk to anyone who started a citizen journalism site or community. They’ll say, “Okay, it took over my life, then it took over my wife’s life, and now we actually have to make money and put food on the table, so we sort of tried to get interns, but we can’t sustain it.” And in terms of the Public Insight Network, we made this commitment at the offset that we could contact everyone at least once a month. Well, so, two things either have to happen in it if — right now the network grows at 2,000 sources a month without any real effort on our part. That’s just simply with outreach and the spread of information.

Miller: And neighbors sharing with friends.

Fantin: Right. And so the idea is that you either have to increase the number of callouts, or you have to increase the size of the cohort that you make the callout out to. Neither of those is really great. But what’s it allowed us to do is realize: “Hey, what we really ought to do is give more control to the source and let them pick and choose and not actually have them sitting passively, waiting for us to ask them questions.” So sometimes, the problems you bump up against help you see where you need to go and you might’ve not known that was the path you were on.

Miller: As we have launched the Public Equals Online campaign, I mean, I think there are all kinds of questions about a community: How do we nurture a community? How do we let them do their own work? Is there top-down control about what they do? What degree? How much can we let go and still have it operating in a single campaign framework? And we all figure these things out as we go along, and no doubt we’ll make mistakes out of it.

Fantin: Well, Amanda Michel at ProPublica, I think, she’s very open about what she learned from her work on OffTheBus with Huffington Post, and now what’s she doing now with ProPublica, which is sort of use citizens to help do investigative journalism, and what you find is that it’s very, very hard. It’s hard because there is a certain amount of information that you can teach people, but on the back end the fact-checking and other things that have to go on in order to make sure that there’s integrity in what you’re reporting. And I’m not here to say that there’s integrity behind what every paid reporter does now. It’s just that this idea we’re gonna have a citizen corps of journalists — or an army of journalists, who for free, are gonna go out there and do the work that people are doing now is—

Miller: It’s just not quite that easy.