Google doesn’t do that, which is in my opinion part of their value. They don’t make semantic assumptions (remember, the small Google News team would have to do this for 40 or so languages) and they’re in the business of expressing what everyone else thinks, not voicing their own opinion.

If I were assigned the project of algorithm development for the Spotlight section, I’d gather data like the time distribution of comments and new links. If people are still commenting on a story at the same (high) rate as they were five days ago, it’s probably not a flash-in-the-pan bit of spot news.

The thresholds could even be calibrated automatically by examining the typical distribution of comment/link production for a news story and looking for long-lived outliers.

I’m not saying this how Google does it, but this is the sort of thing I’d experiment with.

BTW, I suspect your understanding of how article categorization works is similarly off. I would do it by generating word vectors for stories (See http://en.wikipedia.org/wiki/Vector_space_model) and comparing them to content known to be human-categorized in specific ways. Again, this is both category and language neutral.

As a professional computer scientist, I maintain that journalism has a lot to learn from computational linguistics ;)

– Jonathan