Meta delays its website-crawling business terms to 20 November as the UK CMA steps in

Meta pushes its new business terms back three weeks

Meta has delayed the new Commercial Terms and Self-Serve Ad Terms that would let it crawl websites linked to business accounts on Facebook, Instagram, WhatsApp and Threads. The terms were due to take effect on 30 October 2026. A Meta spokesperson told Nieman Lab on Friday 23 October that the effective date is now 20 November, and gave the same explanation to Reuters and Fast Company, Engadget reported.

In its statement Meta said it was extending the date to allow more time to explain to partners and stakeholders what is changing and what is not, including how Meta uses web crawlers and the fact that content owners will still be able to use robots.txt to signal preferences.

The delay is the first concrete concession since publishers raised the alarm in mid-October, when AI Lookout reported that Meta's new terms would let it crawl business websites and originally mentioned training. It is a pause, not a withdrawal: according to Nieman Lab, as of 23 October Meta had made no other changes to the text apart from the date.

What the terms would allow

The disputed clause gives Meta the right to use web scrapers and automated tools to access, index, cache, analyse, collect, retrieve or crawl the domains, web pages and URLs associated with business accounts, and the content available through them. Virtually every news organisation and most businesses that advertise on Meta's platforms have at least one such account.

The first version also said Meta could use that content to provide, develop, improve, research, train and maintain its products. After publishers objected, Meta removed the word train on 16 October and limited future use of scraped content to advertising and business products, Nieman Lab reports. Critics say that still leaves a very broad licence, because Meta's advertising and business products increasingly run on its AI models.

There was no way to opt out of the terms other than to stop using Meta's business tools, which for many publishers and small companies is not a realistic option given how much of their traffic and advertising runs through Facebook and Instagram.

What Meta says it will explain

Meta's position is that little has really changed. On 17 October a spokesperson told Nieman Lab that the initial consolidation of terms had not made clear that robots.txt preferences are respected, and that news sites, like all other content owners, can still disable the crawler through robots.txt. That has been the case, and it has not changed, the spokesperson said.

The gap is that none of the published versions of the terms actually says Meta will respect robots.txt. The promise lives in spokesperson statements and in Meta's crawler documentation, not in the contract businesses are being asked to accept. Publishers have asked Meta to write that commitment into the terms themselves.

Meta has not said what, if anything, it will change in the text before 20 November. Its statement frames the extra three weeks as time to explain rather than to renegotiate.

Regulators and publishers turn up the pressure

In the UK, the Competition and Markets Authority sent Meta a formal letter about the terms, and its executive director Will Hayter told a parliamentary committee he had a meeting with the company scheduled. A CMA spokesperson said on Friday that the authority had been in urgent discussions with Meta, that Meta had now paused implementation until 20 November, and that the CMA would keep scrutinising the terms. The regulator already has new powers over big tech firms, which it is also using in cases such as its proposal to give Microsoft strategic market status over business software and Copilot.

Publisher groups have been lobbying on both sides of the Atlantic. The News/Media Alliance in the United States wrote to Mark Zuckerberg demanding that the terms be suspended before 30 October, and the World Association of News Publishers sent a similar letter to Meta's chief global affairs officer Joel Kaplan. The Alliance warned that publishers would be forced to provide their journalism to support products that may compete with them, under duress and without compensation, Engadget reported.

In Europe, several press associations wrote to the European Commission citing possible breaches of the Digital Markets Act, and two German publisher associations asked the Commission to block the terms until a full review is done. Engadget adds that affected businesses have filed complaints with antitrust authorities.

Why lawyers are still skeptical

Lawyers quoted by Nieman Lab were not reassured. Matt Topic, a media and intellectual property attorney at Loevy and Loevy, said publishers should assume Meta is using the time either to find a new way to win free access to their content or to give up under pressure, and that what they need is for Meta to keep its hands off their copyrighted material. Cornell law professor James Grimmelmann said repeating the same non-explanations for another three weeks would not make much difference.

German competition lawyer Thomas Höppner went further, calling Meta's conduct a prima facie infringement of EU competition law and a textbook case for interim measures. He suggested the delay could be tactical: courts may refuse urgent relief if affected parties wait too long after learning of a change, and some jurisdictions require platforms to give business users proper notice first.

The dispute lands while Meta is already defending how it gathered AI training data in court, including an order that it name everyone who approved downloading pirated books for Llama. That history is part of why publishers are unwilling to rely on verbal assurances.

Robots.txt is doing a lot of work here

Meta's defence rests on robots.txt, the plain text file that tells crawlers which parts of a site they may visit. It is a long-standing convention, but it is voluntary, and Engadget notes Reuters has previously reported that several AI companies' crawlers regularly ignore it.

There is a second problem. A robots.txt rule is all or nothing per crawler, so a publisher who blocks Meta's crawler to keep its articles out of AI products may also lose the link previews and ad features that rely on the same bot. Publishers want the right to say yes to one use and no to another, which is a question of contract terms, not of a text file.

Europe is also debating how much personal and published data AI companies may use without consent, as in the recent push by EU governments for a GDPR change on AI training. Meta's terms are a private version of the same fight, settled by a platform's contract instead of a law.

What publishers and businesses can do now

For now, nothing changes on 30 October. Businesses with Meta accounts have until 20 November before the new terms apply, unless Meta revises or withdraws them. Publishers that want to limit crawling can review their robots.txt rules for Meta's crawlers and check Meta's published crawler documentation, while keeping in mind that the terms themselves do not yet promise to honour those rules.

Trade bodies are urging members to keep the pressure on regulators. Theo Bamber, chief executive of the UK's News Media Association, called the delay welcome but a pause, not a solution, and said the CMA and other regulators should secure clear, public commitments from Meta to drop the offending parts before they take effect.

What happens before 20 November

Three things will decide how this ends. The first is whether Meta writes a robots.txt commitment, or a narrower scope, into the terms rather than leaving it to statements. The second is whether the CMA or the European Commission moves to formal action before 20 November. The third is whether any publisher or business group goes to court for interim relief, which Höppner suggested the delay may be designed to make harder.

The outcome will matter beyond Meta. If a platform can make crawling of partners' websites a condition of using its advertising tools, other platforms with large business user bases could follow. If regulators force changes, it would set an early marker for how far platforms can go in gathering AI and advertising data through their terms of service.