Tom Curley, president and chief executive of The Associated Press, was in China last week for a government-sponsored media summit, where he compared digital content to NCAA basketball and explained the AP’s plans to build revenue online. But Curley was far more revealing when he spoke without a prepared text on October 6 at the Foreign Correspondents’ Club in Hong Kong. I wrote about the big news from that talk on Friday but can now share the audio and transcript.

For all the criticism of Curley and the AP, he had a few really smart observations about the economics of news. Regarding the AP’s competition, which ranges from free news sites to CNN’s new wire service, he was realistic:

Our pricing has to be competitive. There are going to be more competitors. There are going to be fewer people who can afford us. This is a moment of tyranny in the marketplace. There are quality providers, and there are those who aren’t going to be able to sustain the revenues. We don’t expect to have the market share that we used to have.

Speaking broadly about the market for journalists and journalism, Curley was candid:

The truth is, again, the market for news is growing. But the reality is — and none of us can create some fantasy picture here — there is an oversupply, at least in the short term, of us. And so that is creating some differences in the market, and I see these being resolved by innovation and creativity over time.

Oh, there were silly remarks, too, like his misuse of the word “crowdsourcing” and his flat declaration that “we’re all done with random search.” (If that sounds like an ad for Bing, consider it another sign of a looming partnership between Microsoft and the AP.) Others will be interested in his explanation of how the AP will change the licenses for online distribution of its content.

Some comments that I reported on Friday aren’t contained in this recording, which covers Curley’s opening remarks and a question-and-answer session that followed. (Those additional comments are from a separate small-group chat with Curley after his speech, and my source has asked me not to post that audio.) You can download the audio, listen to it below, or read the transcript after the jump.

[audio:http://www.niemanlab.org/audio/TomCurley.mp3]

Here’s the full transcript:

Thank you, Tom. I must say that I feel particularly emboldened to be introduced by someone from The Economist. I have been interviewed by The Economist twice this year, and nothing has made The Economist yet. So I’m feeling I can get away with whatever I want. Secondly, I understand that Gov. Palin has been in Hong Kong recently, and I further understand that she was paid something like $300,000 to appear in Hong Kong. Whatever happens today, you’re getting a bargain. [laughter]

You know, the attendance at journalism sessions and press club meetings these days is a record just about anywhere in the world, and that can be for only one reason. Well, maybe two. One, you’re coming to see how long you’re going to be employed. And, two, is there any hope? I can’t answer the first one, except for the people at the AP, and they’re fine. I can give you a little hope.

And I come with a message of optimism. I must say that for the first time in a decade, when the full sweep of the Internet impact became clear, I now see how we can get to other side, and I now see the leaders of the media organizations having resolved to get to other side. And so I’d like to talk to you in that regard today. No one is promising streets paved with gold or an easy on-ramp to the digital era. In fact, we have only just begun when it comes to figuring out what we have to do. But there is a sea change in attitude, and it starts with the realization that goes like this:

First of all, more people are seeking news more times a time in more places than ever before. And if you understand that, you understand that the market for content is growing. At the same time, the value of that content has been devalued. It’s now at the lowest level, I think, in history. And that’s what we’re about to change. And people have announced a number of different approaches, and many are beginning a number of experiments around what’s generically called pay models, some of which will work and some of which will go to the experimental dustbin.

But the reality is that all of us know that our content is valuable, and what we do, and the sacrifice that you make in asking the tough questions of people around the world, and the risks that people take, especially those front-line journalists, you’re now seeing a response. We deserve to be paid, and now it becomes a manner of trying to figure out how to do that. So the first change is an attitude change, and I can tell you that the CEOs of the media companies — in the United States, at least — have decided to stand up and say, this is the moment. We are not going to be taken to the cleaners. Our content is not going to be monetized by others, and it won’t be. We intend to participate in that stream, in that revenue stream.

The second is the new business models. People are willing to change, and now there is talk of several different business models, and I’ll be talking about some of those ways in a few minutes.

The third is that there is a willingness to do research and experiment, and there are serious budgets for research and experimentation, even during this dire economic setback and even given the fact that resources are strained. So they know that they have to change, and they intend to change right now.

It’s been said that the big mistake that we made, the orginal sin of the Internet, was to give our content away for free. That is not true. We have to engage. The public is consuming content on the web. We have to be a part of that or we will become irrelevant. That was not the mistake we made.

The mistake we made was not imagining more than one product. We took all the good stuff we created, whether for broadcast or for newspapers or even a couple of magazines that have the good sense never to quote me [laughter], and then took it all and put it on the web without much thought. Oh, the number of links, the lightness or heaviness of the page, how much multimedia. Oh, we went through handwringing sessions over that. We didn’t have enough multimedia. The links were too heavy. Or we had too many pictures, and it wasn’t loading fast enough. We spent endless amounts of time over those issues, and they were largely all irrelevant. What we should have been thinking about is a whole different stream of products. And so we are. And so that’s what you’re going to see evolve over the next few years.

You’re going to see what I consider to be a base. I know when it comes to AP, what we’re talking about is that there will be a base license. And you want our content, you display it, use it in your regular product, that’s fine. And display it on the web, that’s fine, that’s one price. If you want to take it and use it with syndication, if you want to use it with email services, if you want to use it and try to draw traffic through other aggregation plays that you’re involved with, there is an upside. And we expect to participate in that upside. So that would be one other way to go.

There are a number of premium products that can be imagined or content verticals, especially, seem to be robust areas of opportunity for growth. And, finally, price can be reserved, and there can be exclusives given, perhaps on a time-based measure, those who get access to that content and the rich multimedia or the metadata that comes with it, might get an exclusive for, oh, 20 or 30 minutes. And we think all those things have value. And those are the product streams that we’re imagining and beginning to create, some of which will be rolled out next year.

The key to that is what we call a news map or a guide to the authoritative source of news, which is to say, who breaks the news? You know, we’re all done with random search. Random search has worked well for the aggregators and the certain portals who have taken our good work and run off with it and given us pennies on the dollar and not paid us appropriately. We’re also done with the people who have taken our content and done crowdsourcing with it, put us in with others that have destroyed our brand reputations and have made these series of links that have been part of the self-referring loop and taken the audience away from the content sites that are broken.

AP members, more than 1,400 of them, have already agreed and are participating in a news registry that will work with editors as well as technology. It will still be curated by editors, and we will provide the links on a real-time basis to breaking news. If a story is broken by The Financial Times, we will take you to The Financial Times website, not The Times of India, even though the Times of India may optimize for search engines more so than The Financial Times of London.

And so that ability to create a news registry is one of the base steps, and we now have, as I said, more than 1,400 organizations participating in that news registry. We will launch and test the first product from this news map and the news registry in six weeks with ten organizations — nine newspapers and one sports statistics provider, a joint venture we have with News Corp., called STATS. And we are creating a news registry.

The key to what we have done is you have to begin by protecting your content. We, especially the United States, have been asleep. We could have done this in 1977, when a copyright law was passed, but we had natural protection by virtue of geography. And so we didn’t have to create copyright protection. The Digital Millennium Copyright Act of the late ’90s came and basically enabled Google and the Google wannabes to do what they are doing, and it’s time for us to take back the night. And it’s time for us to get control of our content, and so we shall do that.

We will put certain marks on all the content that’s working with us. They will include protocols: We will work with you, we won’t work with you. You’re licensed for this, you’re licensed for the full stream. And we will, again, make that available widely to all the participating news organizations. We just had a world summit of news agencies and agreed to extend it to them as well as all the content organizations.

We call this program at AP, 3P: protect, point, and pay. The protection part is to put in the rules that govern the content. These content markers can be stripped, but when the marker is stripped, it calls home, and it tells us, if somebody is stripping the marker on a regular basis, we can then act, either with protocols that restrict the content going there or legally, through legal means. And we intend to set up a service bureau that monitors the usage of content, when it’s downloaded, how it’s used and as well as whether the content is being used in a licensed or unlicensed fashion.

The next part is the point and to point to those who are breaking stories. So if you’re a part of an organization that is breaking stories, you are going to be rewarded. You are going to get the traffic that you deserve because your reporters have been aggressive enough to break the stories or to write in such a way or to produce multimedia in such a way that your content is attractive. It will come to you. It will not go to the free riders. It will not go to the others.

And finally, is the pay piece. And the pay piece is pretty exciting. So if you put the new products with new ideas in terms of how to go to market, you can begin to see that this market for content that is growing might be a market where we, the providers, the producers, start to get paid one way or another, through licensing, through behavioral contextual advertising, through just a fee for having access to that content. Or fee for services, for providing all these backend services that we’re providing, we expect to be able to get some fees for our services.

And also through our own self-referring network. If you go on to Google, normally the top 10, of the top 10 items appearing on Google, 9 of the top 10 will be part of a Google self-referring network. Everything in there is linked to some other part of Google. It’s really a brilliant business approach, and God bless them. And all that money has come to them, all 22 billion. Folks, they can share.

And now there’s something called competition. And I think we stand at an enviable moment where Microsoft and Google have decided to go to war, and we who produce content can start to figure out whether there’s an opportunity for us to help that sharing in a way that reverses the outflow of money from media and takes it back. And so that’s the efforts that we expect to see unfold over the next, especially, 3 to 15 months, and iI think you’ll see a lot more efforts and a lot more announcements that are a lot more bullish from media companies than you have in the last decade. And so that’s the reason for my optimism.

The other part is that we don’t expect to do any of this alone. We expect to do this with partners. And we expect to set up partnerships that we hope are enduring but at least will follow certain principles. We expect real-time metrics. We will no longer work with people who won’t work with us in real time.

Secondly, we are not talking about being luddites, throwing up the moat, and hoping to hold on by taxing negatively consumers who might try to access our content digitally. That is not what we’re doing. We fully understand we have to engage with the new audiences. We fully intend to engage. And we fully intend to use all the tools and ideas of the moment.

AP, in July, conducted an experiment during the Supreme Court nomination hearing for Justice Sotomayor. We began a blog. A blog is kind of an old idea, but we put a couple of twists on it. We worked with Twitter to help get the feedback. We took questions from consumers that they want answered of senators or of Justice Sotomayor. And when the question made sense, we tried to get it answered. We worked with Yahoo as a base, as a center of gravity. We don’t have our own commercial website, and we don’t see going there. But we do know we have to engage with consumers, and we do know we have to engage in a way that is up to what they want on a real-time basis. So we put those pieces in place.

We got a dozen newspapers to work with us, including The Los Angeles Times and The Washington Post, among them, and we ended up driving more than double the traffic that these news organizations have seen as a result of this experiment. Yahoo’s traffic was 27% greater than it normally is, and the traffic to the Sotomayor blog on AP was three times greater than the traffic to the regular Sotomayor story that AP did. So it is possible for us old organizations, 146 years old, in our case, to be reborn and to understand these techniques and to drive the audience.

What we need to do, then, is get paid. We estimate that if we were able to monetize, get the revenue we deserve from all that, then that would have exceeded $200,000 for one week’s extra effort. So that’s the type of thing we have to capture. and we expect to do that with partners, and we expect to do this on an ongoing basis.

Very soon AP will create a new desk called the nerve center. It will be a desk that engages with social media, and we expect to be changing our filing protocols. Filing protocols are a set of standards by which AP copy is filed around the world. We have 15 editing desks, and we expect everybody to get the news out under a certain set of protocols. Right now we have a protocol system. It’s AP, it’s got to be simple: one, two, three. One is to get a headline out. Two is the first paragraph of about 135 words, written in broadcast tense, not newspaper tense. And then three is whatever else you do with the story. We’re going go to zero to four. Zero will be borrowing from broadcast and have a promotional element to it, so that we will start promoting even before we move the story. Four will be a full multimedia effort. So these are the things that we see doing. We see them on an ongoing basis, and we see all our editing desks being empowered to work with these techniques and having that work through a new desk called the nerve center, so that we will engage with the social media networks.

Finally, while I’m here, I also want to say something that I think is really important when you come to the Foreign Correspondents Club of Hong Kong and for all of us when go out in our work. And that is, in this era, we really have to stand up and understand that we have to take up the cause even more vigorously for a free press everywhere we can. You know issues here far better than I, so I will not belabor them, but I do want to call attention to, one, a growing trend, especially among sports leagues, to try to take control of the venues in a way that takes the content creation away from us. We have really got to work together. The big guys in sports have all organized. They’ve gotten their lawyers. And we in the media have organized as well to fight these efforts, and we have to take whatever means necessary, including going to court. We have been to court a number of times, and we have been successful. But this is an area where we have to start guarding. So whether it’s government or sports leagues, at a moment where the digital world is opening up opportunities, it’s opening up, also, some possibilities for those who would take back transparency and put it back in the shadows. and so we need to campaign.

And the second, leads me to this. I urge all of you to come up with tactics that makes sense. What we have decided to do, as a coalition of news organizations in the United States, is to try to get one thing done a year. Pick one thing and get it done, get it fixed, put the spotlight on it. So it’s not a generic call for sunshine or a generic call for the free press. We’re all happy to do that, and we should. But let’s be effective, too. Let’s figure out what we have to do to get something done because at this moment, the stakes are high, and it’s too easy for the pretenders to walk away and say, oh, it’s OK. So all these things, we have to double-down. This is no time — now the economy is tough, there are a lot of excuses that we can have — but now more than ever, every journalist at every level has to double-down and figure out how to be effective and to get it done.

With that, I thank you and, again, I’m delighted to be a part of this. And that Sarah Palin check can come any time. AP’s running a little under budget this year. Thank you.

Question and Answers

[Question about news websites like the BBC that provide news for free.]

Well, here’s what’s going to be different. Anybody who wants our content is going to have to agree to our principles. So we will not grant a license unless the licensee agrees to participate under the terms and conditions that we want. So further re-syndication will be restricted, and we’ll be working with all the large organizations to do it in an intelligent way. And, again, this is not about throwing a moat up and trying to hope that the Internet world passes us by. I assure you we are not that stupid. We really want to figure out how you’re going to market. We want to help you go to market, and want to be a part of it. But if you want to take our content, put it in an email, re-syndicate it, frankly, we’re going to ask for additional fees, and we want an upside. So in the past, it’s been a capped or fixed license approach. So I expect over the next couple of years there will be some wonderful negotiating sessions, and I look at some of my sales people over there [laughter], knowing full well who will take the burden of these conversations, tell them how much I appreciate what they do.

But this is a moment, and it’s not just about AP. It’s about all of us. And what you hear when you talk to the media leaders now is that if we don’t do it now, we are toast. So we are going to stand up, and we are going to go for it. And will we have all the answers immediately? I assure you we won’t. Will we make mistakes? I will only speak for myself and say, yes. But we have to try, and try we shall.

[Question about what he says to young journalists regarding job security in the industry, referencing Global Post‘s model of journalists on retainer.]

A: Well, I’m much more optimistic than no job security and deep cynicism. The truth is, again, the market for news is growing. But the reality is — and none of us can create some fantasy picture here — there is an oversupply, at least in the short term, of us. And so that is creating some differences in the market, and I see these being resolved by innovation and creativity over time. Whether Global Post is one of them, that will be determined or not. But I can tell you that in the United States, in cities that have lost large amounts of coverage, that organizations are being formed. Some of them are being supported by philanthropic groups right now. Some of them are being supported by investors with a particular cause in a given area. And so you will see a lot of this experimentation. There won’t be gaps in the market for long. The gaps will be filled. And i would look at how those gaps are being filled.

The problem with the blogosphere right now is that there are only a few that can emerge at a level where it makes sense or they get paid. And I think that will be an area will there be a lot of growth, successful growth. I think the new methods of the Internet technology will do a sorting there, and there will be more people who will be able to make more of a living, before long, not overnight, through some of those efforts. So there will be a sorting out, and I would not advise somebody to just start blogging, no matter how good they are. Speaking again for myself, I benefited by having a couple of tough editors in my life who whipped me and whipped my writing and who guided me in the middle of the night to some better decision-making. And i worry right now that the pipeline for those journalists will be, right now is too small. And then all of sudden, once we start figuring out how to get paid better, there’s going to be more demand, and there will be a bit of a surprise on the upside, and we will be taking people who were not as trained as we would normally have.

So I think there will be some bumps along this road, but I’m pretty optimistic. Writing is a great skill. It helps you in any number of fields. And you really do have to know two things: It helps to be at least bilingual, and it certainly helps to do multimedia and be comfortable with the new cameras which are emerging and the production of multimedia content. So those are some pieces of advice i would give to young people.

[Question about why consumers would go to the AP’s news portal if it “offers less content for more money than Google.”]

Again, I may not be as stupid as I look. We are not creating a portal, we are not creating a site. We expect to be working with others who are investing billions in these efforts. But we expect to be working with people who will work with us, respect our principles. And we expect to be part of a very large ecosystem. The Sotomayor experiment, again, was indicative of the direction that we chose. We did not do it ourselves. We went out, we got a half dozen newspapers to experiment with, and then we had others [?].

She had an important ruling in a case: The New Haven Register joined the network, Miami Herald joined the network. There was interest in this. So what you’re seeing is an ability to put these networks together. We have, at AP, already networked. We have taken an old newspaper cooperative and used mobile as a way to create our first product of the digital cooperative. We have 1,400 newspapers that work with AP. You can customize, you can put in your local zip code, and you can get the local news for that zip code. So you have national, and you have local. These are the types of things we are doing, and we’re doing it with others. We have reached out to the magazine industry. We have reached out to the networks. We are just not looking at this as self-indulgence at all. This is very different. It’s about a network.

[Follow-up question about what he means by the “news registry.”]

It is not a website. The news registry is a governance structure that has rights and protocols and lots of lawyers and technology people behind it. AP has spent a great deal of money creating and developing Internet technology going back to 2003. We used to be a satellite where we took everything and we broadcast that one product the same way to everybody. And we have become a multimedia, two-way database, and so we are able, we now allow editors and journalists to self-program how they get the AP content through the AP database. And we expect, now, with some associations with certain other portals and certain social network sites, to engage. But, again, we will do it with a set of protocols that we will put in place.

You opt in, and you tell us what your rules are. And you can work with AP doing it all under the terms and conditions you’ve set, or you can have the server at your building. Like, I can’t imagine, for instance, that some news organizations like Reuters would want us to have their Internet deal. So we would expect that if they work with us, the server would be at Reuters, etc. So you can customize it on your own or do it otherwise. We expect for those smaller or mid-sized organizations, that we will provide a service bureau, a dashboard of content, and usage metrics that they can use in building a case for their advertising. And we will have legal support for those who want to engage in activities against the free riders or the pirates.

[Question about the appeal of these initiatives to AP members from a member at a China-based news site.]

Well, great, OK, so I would say, here are four things. One, first of all, the quality of the content is everything, and the level of content and the responsiveness and the number of story breaks in terms of market share has grown, not decreased, over the last couple of years. Two, I do think you’ll see these protocols and these approaches being driven internationally, and if you’re breaking stories, you can be part of the index and the curated piece that would drive the self-referring network and traffic to you very directly. Three, I think that we will begin to see more customized products, and there’s no question that Asia, in general, and the larger China would be top of the list there. So it is our hope that our fledgling Chinese-language translation service moves out and that perhaps there are specialty products that can be created there. Finally, not exactly knowing where you’re going with your content, but I think the multimedia opportunities that we’re putting together and the partnerships that we see could benefit and drive additional revenues to you and hopefully to both of us. So I’d be glad to work specifically on those with you.

[Question about how the AP is responding to the “mini-rebellion” among members who are complaining about price while competitors like CNN emerge.]

Jim, we had more of a rebellion from metro editors last year. We have cut prices to broadcasters and U.S. newspapers $75 million. So that’s one response. The other response is that we see all these new business opportunities as something that they can use to drive revenues as well. So our outreach with the news guide, which would— We’re told that the number one thing that they want, that a news organization wants is credit for when it breaks a story. And this solves that problem. Right now, they don’t get credit, so that’s the number one problem that will get solved. The number two thing is they want traffic from the self-referring network. You heard what i said about the Google self-referring network. It works really well for Google. We’re talking about creating our own self-referring network, and we know that the traffic increase will exceed 20% as a base within that network once we put it together. So that’s another aspect for that.

Our pricing has to be competitive. There are going to be more competitors. There are going to be fewer people who can afford us. This is a moment of tyranny in the marketplace. There are quality providers, and there are those who aren’t going to be able to sustain the revenues. We don’t expect to have the market share that we used to have. We are hopeful that we will be able to sustain the relationships among the places that you have suggested by, number one, breaking more stories than anybody and also with a reputation for accuracy and, two, customizing products. We have one network that was raising some questions about our pricing, and we went back, and we looked: AP correspondents appeared on that network 144 times in the last 10 months, and that was a service that they got without being charged extra for. That helped changed that conversation. So, again, it’s going to be customizing for specific needs. That is the era that we live in. It is not one product fit all. It is that multi-tier product system. And I think all of us in media made a mistake a few years ago by not understanding that sooner, and that is what we have to change.

[Question about how the AP is different from Factiva.]

Did you say Factiva? First of all, we create real-time content. We have the journalists out there. That’s archives. So I think the archive providers are going to be challenged by the technology going forward. One of the partners that we’re in negotiations with is going to make a serious play for real-time archives, and one of the new products that we’re talking about is making our database and our historical database, including some video and film that we have going back to 1903, more easily available. I can tell you that one of our partners now has 50,000 dead-end— or 50 million dead-end searches a month, trying to get historical stories from us. That looks like a heck of a market if we opened that up and made it available. So that is one of the major new products that we see, and we think that’s a significant positive revenue stream going forward.

[Question about how consumers will benefit from the AP’s plans.]

Well, this is the moment for the consumer. I think, first of all, John, keep reading. But for the last 10,000 years or so, it has been mass media one way or another, the other way. And certainly, going back to the [?] age and looking at the stories being told there, somebody put the paintings there and drawings for all of us to interpret. We’ve gotten a little bit more transparent since then, but now you’re talking about two-way and choice and being able to self-program and self-edit. Some people are excited by the Kindle. I find the iPod life-changing. And books available. I carry two iPods, and if one of them blows on a trip to China, I have an opportunity, I have insurance, I have a backup. So I really think this is the moment for the consumer.

The downside is, wow, you have to work at it right now. And I guess I would say my hope is there would be some editing functions and you will be able to get a richer source of information that you want customized for you for a bit of a price, maybe less than what you’re paying now, maybe a little bit more, depending. But you will be able to get the stuff that you want, that’s more driven to [?] and something that you create. And I think that’s the big payoff.

[Question about Curley attending a conference hosted by the Chinese government and the implication of that decision on the AP’s support of a free press.]

So you can take the approach that we aren’t going to talk and we aren’t going to work. I don’t think that’s a very smart approach. I think people know who I am and what I stand for, and iI testified last week at a U.S. senate committee on openness and access to government, and my stands are known. I have given, I think, what was called a very strong freedom of the press speech in China in advance of the Olympics when it looked like they weren’t going to be as open as they were. And so this is part of the Olympic platform that they engage with the world, and this conference is part of that, to keep engaging. I think it’s in all our best interests to keep that conversation going and to stand up for the values that we believe in. You don’t have such freedom to get there, and I hope to have the opportunity to figure out how to say that so that you can travel more freely and that our people can travel more freely. Looking at what’s happened in China since 1972, when AP reopened its bureau, things are far better in 2009 than they were in 1972. I am pleased to salute China for its total progress, and if you look at what’s happened in the last 60 years, there are a lot more people in China living better, and they deserve a lot of credit for that.

And so I think that, I hope to be a good guest, but again, I will look for the one opportunity to say what has to be said. We’re not fully there. And I say that in the United States. It’s a good conversation to have. We have governments in the United States that would like to keep more things secret, a lot more things secret than they should. And so it’s a battle to be fought everywhere in the world by all of us.