Apple rewrites App Store rules to make AI apps block nonconsensual nude images

What Apple changed on 16 October

Apple has rewritten part of its App Review Guidelines so that apps which let people create or share content must actively keep nonconsensual intimate imagery off their platforms, and it now says in writing that this includes images made with AI tools. The change was announced on 16 October in a developer news post titled "Strengthening App Review Guidelines against nonconsensual intimate imagery and bait-and-switch apps", alongside a routine changelog for the same revision.

Three changes carry the weight. Guideline 1.2, which governs user-generated content, now names nonconsensual intimate imagery and AI content creation apps explicitly and warns that developers who fail to put safeguards in place can be found in breach of the Developer Code of Conduct. A new Guideline 2.5.19 spells out the ban on hidden or dormant features, which Apple ties directly to so-called nudification apps that pass review and then switch on their real purpose. And Section 5 now says that apps facilitating human trafficking or the exploitation of children will be removed from the App Store and blocked from alternative distribution.

The same revision added a Guideline 2.3.14 on creative assets used in search results and product page headers, merged a bullet about music streaming entitlements into the StoreKit External Purchase Link bullet in 3.1.1(a), and added documentation links before and after submission. Apple says translations will follow within a month.

The new wording in Guideline 1.2

The clearest way to see what moved is to compare the guideline text. In a Wayback Machine copy from 15 October, Guideline 1.2 began: "Apps with user-generated content present particular challenges, ranging from intellectual property infringement to anonymous bullying." The live guidelines now read: "Apps that enable users to create, generate, upload, share, or otherwise contribute content present particular challenges, including intellectual property infringement, nonconsensual intimate imagery, and anonymous bullying. This includes social media, user-generated content platforms, and AI content creation apps."

That rewrite matters because it widens who the rule plainly covers. The old text spoke of apps with user-generated content or social networking services. The new verbs, create and generate, reach an app whose users never upload anything but type a prompt and receive an image. An AI photo editor or a face-swap tool now sits squarely inside the same moderation duties as a social network.

Those duties did not change. Such apps must still include a method for filtering objectionable material before it is posted, a way to report offensive content with timely responses, the ability to block abusive users, and published contact details. What is new is a sentence about consequences: "Developers who fail to implement appropriate safeguards or fail to remediate identified violations after notice may be considered in violation of the Developer Code of Conduct." The guideline already said Apple would ask for removal and a compliance plan, and that "egregious or repeated behavior" could mean immediate removal from the App Store and the Apple Developer Program. In its news post Apple describes the duty as content apps "must proactively prevent and remove", which is stronger language than the guideline itself uses.

Bait-and-switch apps get their own rule

Apple's post names the tactic it is targeting: apps that "misrepresent their purpose during review and only reveal harmful capabilities, such as nudification, after approval, a tactic commonly known as bait-and-switch." A typical pattern, according to the reporting Apple's move follows, is an app that presents itself as a harmless face-swap or photo tool and only shows its sexual image styles once a person is using it.

The new Guideline 2.5.19 reads: "Don't include any hidden, dormant, or undocumented features in your app; your app's functionality should be clear to end users and App Review. Attempting to hide undocumented features in your app binary, such as through code obfuscation or other methods, is grounds for removal of your app from the App Store and termination of your developer account." The first sentence is not new. It opened Guideline 2.3.1(a) in the 15 October copy and has now been moved out on its own. The second sentence, which names code obfuscation and spells out the penalty, is the addition.

Apple's post adds that the rule also covers apps that change after approval, "for example, adding features or functionality that weren't included in the version of the app that was submitted to App Review." That is relevant to AI apps in particular, since a model or image filter running on a developer's server can change what an app does without any new binary reaching review.

Removal from alternative distribution, too

The third change is short. Section 5 previously said that in extreme cases, such as apps found to facilitate human trafficking or the exploitation of children, "appropriate authorities will be notified." It now adds that "the apps will be removed from the App Store and blocked from alternative distribution." The guidelines use alternative distribution for apps installed outside the App Store where Apple permits that. Apple's post says the App Store "has always prohibited" such apps and that the update is meant "to make clear" the outcome, including for child sexual abuse material.

Apple also gave scale figures. It says that last year it prevented more than 150,000 apps with safety issues from reaching customers and terminated more than 193,000 developer accounts for fraud and abuse, some tied to "coordinated fraud operations" that hide harmful content until after approval. Apple did not break those numbers down by category, so they say nothing specific about nudify apps.

Why Apple is doing this now

Apple's post does not mention any outside report, but the update follows a year of pressure that 9to5Mac summarised on the day. In January the Tech Transparency Project said it found AI "nudify" apps easy to locate on both the App Store and Google Play. According to 9to5Mac's account of that report, 47 iOS apps and 55 Android apps complied with its test requests, which used AI-generated images of fictional women, and the apps it identified had been downloaded more than 705 million times and earned about $117 million, figures the group took from the analytics firm AppMagic. At least one was rated for ages 9 and up.

A follow-up in April said App Store search suggestions and ads were steering users to such apps, and that nearly 40 percent of the top ten results for searches like "nudify" and "undress" could render women nude or scantily clad. Apple told 9to5Mac then that nudify apps already broke its rules and that it had removed 15 of the apps named. In July, San Francisco City Attorney David Chiu sent cease-and-desist letters over 13 apps, eight of them on the App Store, and Apple said it had removed three and was terminating their developers' accounts while it pressed four others, according to 9to5Mac's July report.

The Grok episode also hangs over the change. Early in the year, users found that xAI's chatbot would readily undress people in photos, including women and minors. A letter Apple sent to US senators, reported by NBC News and summarised by 9to5Mac, said Apple found X and Grok in violation of its guidelines, rejected a Grok update as not going far enough, and approved a later version only after further changes. The new 1.2 wording, with its reference to AI content creation apps and to remediating violations after notice, reads very much like a written version of that process. For more on how AI-made images are reshaping trust in what people see, see our report on OpenAI and Meta's warnings to a federal evidence panel.

What developers of AI image apps should do

For most developers nothing new has to be built if their moderation was already real. For anyone shipping a general AI image or video generator, an editing app with generative fill, or a face-swap tool, the practical reading is that App Review can now treat weak safeguards against sexual images of real people as a conduct issue, not only a content issue. In practice that means filtering prompts and outputs that sexualise real, identifiable people; a working report button with a real response; account blocking; and a contact address.

The bait-and-switch rule also argues for documenting server-side model changes. If an app's image model or filter settings are changed remotely, the safest course under 2.5.19 and 2.3.1(a) is to describe that behaviour in review notes rather than let reviewers discover it later. Apple says obfuscating features in a binary is grounds for account termination.

Detection tools are improving alongside policy. Google's watermark checker, covered in our story on the public SynthID Detector launch, is one way platforms can spot AI-made images, though watermarks only catch images from models that add them. Apple's own AI plans in other markets are covered in our piece on Apple Intelligence in China.

What the update does not say

The update leaves several things open. Apple has not said how App Review will test AI apps for nudification before approval, whether it uses automated scanning of outputs, or whether existing apps get a deadline to comply. It has not said whether general-purpose chatbots with image tools will be judged by the same standard as single-purpose face-swap apps, although the new text names AI content creation apps without exception.

The figures behind the pressure campaign come from advocacy and press reports, not from Apple, and Apple has not published how many nudify apps it has removed in total. Google Play's policy already bans apps that claim to undress people, according to the January report, so this brings Apple's written rules closer to Google's rather than past them. Whether it changes what people can actually find in the App Store will depend on enforcement, which outside researchers will be able to test with the same searches they ran this year.