Skip to contentNewskeryx

31 October 2026

Four AI firms back one robots.txt opt-out from AI training

First reported

Engadget, SiliconANGLE and 1 other ran this on , all on the same day.

  • Google, Microsoft, OpenAI and Anthropic will honour a shared website setting that blocks AI training scraping.
  • Sites currently must add separate rules for each company's crawler, which the new single signal would replace.
  • Support arrives in stages: Anthropic is working on it now, OpenAI in coming months, Microsoft February 2027, Google early 2027.

Reported by The Next Web, SiliconANGLE, Engadget