Google and Microsoft announced on October 30, 2026 that they will honor a joint opt-out from AI training in robots.txt. To do so, website operators enter the new identifier __usage_ai-training in the User-Agent line and block beneath it the paths whose content should not flow into the training of generative AI models. According to the Bing blog, Microsoft will take the new entries into account when crawling from February 1, 2027.
In the Search Central Blog, Google names 2027 as its target, and the crawler documentation speaks of early 2027. According to SiliconANGLE, OpenAI also intends to honor the entry in the coming months. The draft was submitted to the Internet Engineering Task Force (IETF) by Kevin Kelley of Anthropic. The IETF lists the paper as an individual's draft without official status.
What does the entry in robots.txt look like?
To opt out of training on all the content of your website, two lines are enough:
User-Agent: __usage_ai-training
Disallow: /
You release individual folders again with Allow. According to the draft, training is any change to the learned parameters of a generative AI model; models that merely classify or sort content do not fall under it. A rule under User-Agent: * explicitly does not count as an opt-out.
Crawlers may still fetch your pages, because the identifier governs the use of the content, not access. Microsoft assures that the new entries do not affect Bing search.
What changes at Google and Microsoft?
Google builds on Google-Extended, its own identifier against AI training since 2023. If a website blocks training via the joint entry and does not mention Google-Extended separately, Google treats the entry like a prohibition for Google-Extended. A separate rule for Google-Extended takes precedence. According to Google's documentation, Google-Extended controls two things together: the training of future Gemini models and grounding in the Gemini apps, that is, answers using content from the search index.
When the entry takes effect
Joint opt-out from AI training via robots.txt, timeline from Microsoft and Google
- October 30, 2026Announcement and draft at the IETF
- February 1, 2027Microsoft takes the new entries into account when crawling
- March 1, 2027NoArchive and NoCache no longer count as an opt-out at Bing
- AnnouncedGoogle plans to read the entry in early 2027, no fixed date
The standard is a draft, providers honor the entry voluntarily
Sources: Bing Blogs, Google Search Central, IETF Datatracker
Microsoft separates training from grounding, that is, retrieving a page to support an answer in Bing, Copilot or Microsoft 365 with source links. You block training via robots.txt or page by page with the meta tag <meta name="Bingbot" content="ai-training:disallow">. For grounding, only the page-by-page entry ai-grounding:disallow in the meta tag or in the HTTP header X-Robots-Tag remains.
Until now, at Bing the NoArchive and NoCache entries counted at the same time as an opt-out from use in Copilot. After March 1, 2027, Microsoft will no longer evaluate either of them for this. As a transitional measure, from December 15, 2026 you can set in Bing Webmaster Tools whether Bing may use your content for grounding, limited until September 30, 2027.
What this means for you
At Google, the entry contains a conflict of goals for your website. If you block training only via the joint entry, then under Google's rule you also exclude grounding in Gemini and probably no longer appear there as a source. A separate release for Google-Extended keeps you visible, but at the same time allows Google to train. The course on visibility in ChatGPT, Claude and Gemini shows what role AI answers play for your visitor numbers.
If you currently use NoArchive or NoCache against Copilot, note down March 1, 2027. Until then you need Microsoft's new meta entries, otherwise grounding is permitted again.
WordPress generates robots.txt virtually as long as no real file is in the root directory. With the file editor of Yoast SEO you create the file and add the two lines there; how to set up the plug-in is shown in the lesson Configuring Yoast SEO.
The entry is binding for no one. Providers that do not take part do not read it, and according to SiliconANGLE Meta supports the idea without promising implementation. It is therefore best to leave your existing rules for individual crawlers in place.
Google and Microsoft published their plans on October 30, 2026.
Sources
