Google has been using publisher content for AI Overviews — even if publishers have “opted out” of being used to train Google’s Gemini model, according to testimony from a Google product VP.
Bloomberg’s Davey Alba first reported on the testimony, which was part of ongoing antitrust hearings in the D.C. federal circuit court to determine how best to break up Google’s general search monopoly.
Bloomberg’s recounting of the testimony here:
“Once you take the Gemini” AI model “and put it inside the search org, the search org has the ability to train on the data that publishers had opted out of training, correct?” asked Diana Aguilar, a Department of Justice lawyer.
“Correct — for use in search,” Collins responded.
Instead, the company willfully uses the data publishers have asked not to be used for model training in its AI-generated responses to user search queries.
Some publishers have claimed that AI Overviews are decreasing their search traffic, since users may not have a reason to click through to an article page if a publisher’s repurposed content appears at the top of search results. Earlier this year, Chegg, the online education company, sued Google for just that reason, claiming it has upended their revenue and business model.
In a statement to Bloomberg, a Google spokesperson clarified that “Google has a separate way for publishers to manage their content in Search via the well-established robots.txt web standard.” The spokesperson is referring to a snippet of code on a website’s directory that tells AI crawlers which parts of a site they are not allowed to scrape. (While it is a norm to respect robots.txt directives, it is a norm that has been repeatedly broken by other AI developers).
Currently, there is no way for publishers to use robots.txt to only block Google’s indexing for AI Overviews, and not general search. So Google’s statement confirms the only way for publishers to opt-out is to effectively block Google’s ability to rank their site in search results altogether.
Another revelation from the Friday testimony was just how much Gemini’s training data was impacted by publishers opting out. DOJ lawyer Aguilar shared an internal Google document from August 2024 in court:
According to that document, Google removed 80 billion of 160 billion “tokens” — snippets of content — after filtering out the material that publishers had opted out of allowing Google to use for training its AI.
You can read the full story on Bloomberg.
This story has been updated to remove an inaccurate reference to Gemini’s total training data.
