🔍 Read the full analysis: A Question-and-Answer Guide To The AI Training Opt-Out on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
OpenAI, Google, Microsoft and Anthropic have said they will support a shared robots.txt signal asking them not to use a website’s content to train generative AI models. It is a voluntary preference, not a technical block or an opt-out from AI-generated answers, and company start dates differ; Meta has voiced support without committing to honor it.
OpenAI, Google, Microsoft and Anthropic said on October 30 that they will honor a shared robots.txt signal asking not to use a website’s content for generative AI training. The proposed rule could let publishers state that preference once rather than maintain separate instructions for each company, but it is voluntary and does not prevent all use of a site’s content by AI services.
The signal appears in an individual Internet-Draft titled Machine-Readable Training Preference Signal, authored by Anthropic standards lead Kevin Kelley and published October 30. The proposed robots.txt group uses User-Agent: __usage_ai-training followed by Disallow: /. The draft describes a preference against using a site’s content to train generative AI models; it is not yet an official internet standard.
Companies have given different implementation timelines. Anthropic says it has started implementing support, while noting that adding it across its services will take time. OpenAI says it aims to implement the signal “in the coming months.” Google aims to support it in 2027; its crawler documentation says “early 2027.” Microsoft says it will honor the shared training preference starting February 1, 2027, as it crawls websites.
Microsoft also announced a separate page-level preference for “AI Grounding,” covering web content used to inform AI answers in Bing, Copilot and Microsoft 365. Its announced controls take effect February 1, 2027. Microsoft says its older NoArchive and NoCache tags stop counting as preferences for training or grounding after March 1, 2027. Meta voiced support for a common signal, but did not say whether it will honor this one.
Infographic 01 · Who backs it, and when
One line in robots.txt, four companies, four timelines.
A shared signal against AI training. Only Anthropic says it has started; the others name dates or months. The standard itself is still a draft.
- 30 Oct 2026Draft published, four companies announce supportdoneIndividual Internet-Draft by Kevin Kelley (Anthropic)
- 30 Oct 2026Anthropic: “has already started implementing support”startedIts standards lead on the IETF list: “It’ll take us some time to add support across our services.”
- Coming monthsOpenAI aims to implement itplanExisting GPTBot and OAI-SearchBot rules keep applying
- 16 + 19 Nov 2026Working group meets at IETF 127, San FranciscoopenOn 21 Oct a chair said no consensus has been declared yet
- 1 Feb 2027Microsoft honors the new preferences as it crawlsplanTraining in robots.txt or per page; grounding per page
- 1 Mar 2027Bing stops reading NoArchive / NoCache as AI preferencesdeadlinePublish the new settings before this date
- 2027Google plans support; its docs say “early 2027”planA “no” maps to Google-Extended unless you set that token yourself
- 3 May 2027Draft version 00 expires unless updatedInternet-Drafts are valid for at most six months
Binding? No, voluntary
- OpenAI calls it a “voluntary effort”.
- The vocabulary draft does not “ensure that preferences are followed”.
- The draft “has no formal standing in the IETF standards process”.
- Private agreements may override it, the draft says.
Who else?
- Meta supports a common signal, but did not say whether it will honor it.
- Apple, Amazon, xAI, Perplexity: no statement in the sources reviewed for this article.
- The draft’s acknowledgments list contributors “for identification only”, not supporters.
Sources: IETF draft, 30 Oct 2026 · Anthropic on the IETF list · AIPREF minutes, 21 Oct 2026 · Google, 30 Oct 2026 · Google crawler docs · OpenAI, 30 Oct 2026 · Microsoft Bing, Oct 2026 · Meta (Andy Stone on X). Company plans are company statements.
Powered by Thorsten Meyer AI
A simpler publisher training preference
The shared rule could reduce the work involved in expressing a training opt-out. Before this proposal, site operators commonly needed separate instructions for different crawlers, such as GPTBot, Google-Extended and ClaudeBot. A common signal offers a single additional way to communicate the preference to participating companies.
Its practical effect depends on company implementation and publisher expectations. The signal is a request, not a technical lock: it does not prevent a crawler from accessing a page, and the announcement does not make the preference binding on every AI company. Microsoft also says that neither choice guarantees inclusion in search or AI answers, traffic, or payment. Publishers still need to distinguish between training, AI answers based on web content, and ordinary search access.
Popular right now
De'Longhi 15-Bar Pump Espresso Machine with Milk Frother & Temperature Control
KitchenAid Artisan 5-Quart Tilt-Head Stand Mixer, KSM150PS, Almond Cream
Sony BRAVIA 5 85-Inch Mini LED 4K HDR Smart TV (K-85XR50)
As an affiliate, we earn on qualifying purchases.
The draft builds on work by the Internet Engineering Task Force’s AI Preferences working group, which considers ways for websites to express machine-readable preferences. The companies’ announcements refer to the same draft, but publication as an individual Internet-Draft does not mean the IETF has adopted it as a standard. At a working-group meeting on October 21, a chair said, “We haven’t declared consensus on anything yet.”
The new signal is intended for website operators, not people who want to prevent their own conversations with an AI assistant from being used for model training. Chat and account data are governed by each service’s own settings and policies. Existing crawler-specific rules remain relevant while providers implement the shared signal; OpenAI says its current controls will continue to apply. Google says the shared opt-out will be treated as if a publisher had disallowed Google-Extended when no Google-Extended rule is present.
““has already started implementing support for this draft””
— Anthropic, in its announcement of support for the draft
Coverage and compliance remain open
It is not yet clear how many other AI companies will honor the signal. The source material reports no announcement from Apple, Amazon, xAI or Perplexity, and Meta has not made an implementation commitment. Support from the four named companies also does not establish how consistently every service operated by each company will apply the preference.
The proposal does not cover every use of online material. It addresses training, not AI answers that draw on live web content; Microsoft’s separate grounding preference is one company-specific control for that use. The signal also cannot remove material from models that have already been trained. Its status in the IETF process remains unsettled, and the companies’ rollout schedules are not identical.
Watch company rollout dates
Publishers can review their robots.txt files and decide separately what they want to express about training, AI grounding and search. They may add the shared rule if they want to state a training preference, while retaining company-specific crawler rules until each provider has implemented the common signal. Sites using Microsoft’s NoArchive or NoCache tags as training or grounding preferences should account for the stated March 1, 2027 cutoff.
The next developments to watch are whether the IETF working group reaches consensus on the draft and whether companies provide clearer rollout details. Anthropic’s implementation is underway, OpenAI’s target is the coming months, and Google’s documentation points to early 2027. Microsoft’s announced start date is February 1, 2027. The source material does not give a date by which all four companies will support the signal across their services.
Key Questions
What does the shared AI training opt-out do?
It lets a website express a preference against using its content to train generative AI models through a common robots.txt rule. It is voluntary and does not technically block crawling.Which companies have said they will honor it?
OpenAI, Google, Microsoft and Anthropic have announced support, with different rollout schedules. Meta has voiced support for a common signal but has not said whether it will honor this one.Does the rule stop AI services from using my pages in answers?
Not by itself. The shared signal addresses model training, not AI answers based on live web content. Microsoft announced a separate AI Grounding preference for certain Microsoft products.Does it remove content from models that are already trained?
No. The proposal expresses a preference about training use; it does not remove content from existing models.Is the signal an official internet standard?
No. It is an individual Internet-Draft connected to IETF working-group activity. The material says the working group had not declared consensus on a proposal.Source: ThorstenMeyerAI.com
