Meta plans to put its new Commercial Terms of Use into effect on November 20, 2026. A clause in them allows Meta to use crawlers to read the websites that are linked to your company's products or services.
The original start date was October 30, as Nieman Lab and Search Engine Journal report. Meta has postponed the launch by three weeks to give partners more time for clarifications.
What does the crawler clause allow?
According to the preview page, you grant Meta or a service provider acting on Meta's behalf the right to use automated tools such as scrapers, bots, and crawlers. The tools may access, index, cache, and analyze domains, websites, and URLs. The clause names the purposes as providing and improving advertising products and other offerings for businesses, as well as security and protection against abuse. Meta may also create test credentials.
According to Search Engine Journal, the terms apply to every business use, including managing a Facebook Page. Nieman Lab mentions business accounts on Facebook, Instagram, WhatsApp, and Threads.
On October 16, Meta removed the word "training" from the clause, Nieman Lab reports. According to Search Engine Journal, the Self-Serve Ad Terms, Meta's terms for self-booked ads, dated August 22, 2025, already allow Meta to crawl content on advertisers' domains for research and training as soon as an advertising order has been placed.
Meta explains that the wording became unclear when the terms were merged. The company says it continues to respect settings in robots.txt, and Social Media Today gives data collection for ad targeting as the purpose. However, the previews of the terms say nothing about robots.txt, as Search Engine Journal and Nieman Lab note.
Does the November 20 start date still apply?
That is unclear. Social Media Today writes on November 2, citing Reuters, that Meta could postpone the rollout indefinitely or drop it altogether. The article names this as a possibility, not a decision. On November 3, the preview page still shows November 20.
The Meta terms timeline
Crawler clause of the Commercial Terms of Use, as of November 3, 2026
- October 16, 2026Meta removes the word "training" from the clause
- October 30, 2026originally planned start of the terms
- November 20, 2026new start according to Meta's preview page
- OpenSocial Media Today: start could be dropped indefinitely
According to Social Media Today, the indefinite postponement is a possibility, not a decision
Sources: Meta, Nieman Lab, Search Engine Journal, Social Media Today
The UK competition authority CMA is continuing to review the terms, and the EU Commission welcomed the postponement, according to Nieman Lab. Publisher associations such as WAN-IFRA are demanding that the crawling paragraph be deleted.
What this means for you
If you continue to use Meta's offerings for businesses after the notification, this counts as consent to the amended terms, including the crawling rights, according to Search Engine Journal. According to Nieman Lab, Meta did not provide for an opt-out.
So take a look at your website's robots.txt before November 20. In its crawler documentation, Meta lists five identifiers (user agents). With meta-externalads, Meta crawls for advertising and other offerings for businesses, with meta-externalagent for training AI models or for indexing, and with meta-webindexer for Meta's AI search. In addition, there are meta-externalfetcher for links that someone has fetched, and facebookexternalhit, which reads the title, description, and preview image as soon as someone shares your page in a Meta app.
One Disallow line per identifier blocks access, and changes take effect within up to 24 hours according to Meta. The documentation itself admits two exceptions: meta-externalfetcher can bypass robots.txt, and facebookexternalhit possibly does so during security checks. Whether a block limits the rights under the terms cannot be determined from any preview. Therefore, block only the identifiers whose purpose you do not want.
Make a note of November 20 and check the preview page the day before to see whether the date still stands. What a robots.txt entry against AI training at Google and Microsoft looks like is explained in the news from October 31. To get started with ads, a lesson from the Paid Ads tutorial shows how to set up Meta Ads.
Nieman Lab and Search Engine Journal reported the postponement on October 26, 2026, and Social Media Today reported the current status on November 2, 2026.
Sources
