Where AI actually sits inside the Online Safety Act

The UK Online Safety Act 2023 was not written as an AI law. It regulates "user-to-user" services and search services, and it imposes illegal content and children's safety duties enforced by Ofcom. AI only comes into scope indirectly, through how content is generated, shared and encountered on services that are already caught by the Act. Ofcom has been explicit that this creates gaps as well as duties, and it has spent 2025 and 2026 issuing guidance, opening investigations and updating its plan of work specifically to address generative AI, chatbots and AI-generated sexual imagery, as set out in Ofcom's strategic approach to AI, 2026/27, which flags "nudification" content and other AI-enabled harms as a live priority.

Ofcom's codes of practice and AI-generated content

Ofcom's Illegal content Codes of Practice for user-to-user services were issued 24 February 2025 and became enforceable from 17 March 2025. The codes do not treat AI-generated material as a separate category. Where a user shares or uploads AI-generated content on a user-to-user service, it is regulated exactly as human-created content would be, against the Act's list of more than 130 priority offences, and services must apply measures such as content moderation and, on higher-risk services, perceptual hash-matching to detect known child sexual abuse material (CSAM), regardless of whether the image is real or synthetic. Ofcom's separate Statement: Protecting children from harms online, published 24 April 2025, set the children's safety codes that became enforceable from 25 July 2025, requiring highly effective age assurance to keep children away from pornography and other primary priority content, again without carving out an AI-specific exemption.

Chatbots and companion apps: the Part 3 versus Part 5 boundary

The most important, and least intuitive, feature of the regime is a scope boundary that Ofcom had to clarify explicitly. Its AI chatbots and online regulation, what you need to know guidance explains that AI-generated content shared by users with other users on a user-to-user service is regulated as user-generated content. But a chatbot that only interacts one-to-one with a single user, does not search the internet on the user's behalf, and cannot produce pornographic content, falls outside the illegal content and children's safety duties in Part 3 of the Act altogether. Ofcom's update on the investigation into X and the scope of the Online Safety Act confirms this directly: a private, one-to-one chatbot exchange that is not search content and is not shared with other users sits outside Part 3, though it can still be caught under Part 5 if the output is pornographic. This is why companion apps and standalone chatbots occupy a genuinely uncertain position, and why Ofcom's own open letter to UK online service providers on generative AI and chatbots, published 8 November 2024, was needed to put providers on notice that features letting users build and share their own AI personas or chatbots do bring a service back within Part 3.

Deepfakes and AI-generated CSAM: enforcement, not new drafting

The Act itself does not create bespoke new offences for deepfakes or AI-generated CSAM. Instead it relies on existing and newly created criminal law as the "illegal content" baseline, and forces platforms to detect and remove it. UK criminal law already treats intimate images of under-18s as illegal whether or not they are artificially created, and non-consensual sharing of an intimate image of anyone, AI-generated or not, is separately a criminal offence. That baseline was reinforced by the Crime and Policing Act 2026 child sexual abuse material factsheet, which introduces offences for possessing, creating or distributing AI models optimised to generate CSAM and for "paedophile manuals" covering AI-generated abuse material. Ofcom's role under the Online Safety Act is to enforce the platform-side duty to find and remove that content once it is illegal, not to define the offence itself. The gap between drafting and enforcement is visible in two live cases: Ofcom's investigation into X over Grok sexualised imagery, opened 12 January 2026 after reports that the Grok chatbot on X was generating sexualised deepfakes of real people, including children, and its investigation into the Novi Ltd companion chatbot service, Joi.com, opened 15 January 2026, focused on age assurance and pornographic content duties. Both are being run under the Act's existing illegal content and age-assurance duties rather than any AI-specific power, which is itself the clearest evidence of how Ofcom is applying the current framework to generative AI.

Ofcom's generative AI guidance, in sequence

As of mid-2026, more than 100,000 online services fall within the scope of the Act, according to reporting on Ofcom's plan of work summarised by the Digital Watch Observatory, a scale that makes case-by-case AI scoping questions a recurring compliance issue rather than an edge case.

Where the UK and EU approaches converge, and where they genuinely differ

The EU AI Act's Digital Omnibus adds two new prohibited practices to Article 5: AI systems that generate or manipulate non-consensual intimate imagery ("nudifier" apps) and AI systems that generate CSAM, both expected to become enforceable from 2 December 2026, carrying the AI Act's top penalty tier of up to EUR 35 million or 7% of global annual turnover, as described by Gibson Dunn and TechTimes. This is a product-level ban aimed squarely at the AI system itself, catching a nudifier tool even where the provider never intended that use, if the outcome is reasonably foreseeable and no adequate safeguard exists.

The Online Safety Act does not ban an AI model or app as such. It regulates the service on which AI-generated content is shared or encountered, and it leaves the underlying question of what counts as illegal content, including AI-generated CSAM, to separate UK criminal law. The two regimes converge on the target, nudification and AI-generated CSAM are treated as unacceptable in both jurisdictions, but they diverge sharply on mechanism: the EU is prohibiting a class of AI system outright, while the UK is placing a detection and removal duty on the hosting platform and relying on criminal law to define the underlying offence. A UK-only nudifier app with no user-to-user sharing function could fall outside Part 3 of the Online Safety Act entirely while still being caught by the EU's Article 5 ban if it is offered into the EU market, and conversely a UK platform hosting user-shared nudified images is squarely an Online Safety Act problem regardless of what the EU rule says. Treating the two as interchangeable compliance obligations would be a mistake.

What this means in practice

  • Platforms offering user-to-user chatbot or persona-building features, including the ability to share, publish or send a chatbot's output to other users, should assume Part 3 illegal content and children's safety duties apply, and should extend existing hash-matching and moderation tooling to AI-generated as well as human-created content.
  • Providers of standalone, one-to-one companion chatbots should not assume they are out of scope by default. Ofcom's own tests, on searching behaviour, on sharing with other users, and on pornographic output, need to be checked service by service, and the Novi Ltd/Joi.com investigation shows Part 5 pornography and age-assurance duties can apply even where Part 3 does not.
  • App stores and distribution platforms carrying AI companion or "nudifier"-adjacent apps face converging pressure from both regimes: UK criminal law and Ofcom's illegal content duties on the hosting or sharing side, and the EU AI Act's outright product ban from December 2026 on the development and deployment side.
  • Any UK service that also serves EU users needs to track both obligations separately. Meeting Online Safety Act illegal content duties does not satisfy the EU AI Act's Article 5 prohibition, and vice versa, so compliance teams should map each feature against both frameworks rather than assuming a single control set covers both.

Primary sources: Ofcom Illegal content Codes of Practice · Ofcom Statement: Protecting children from harms online · Ofcom open letter on Generative AI and chatbots · Ofcom: AI chatbots and online regulation · Ofcom update on X investigation and OSA scope · Ofcom investigation into X over Grok · Ofcom investigation into Novi Ltd/Joi.com · Ofcom's strategic approach to AI, 2026/27 · Digital Watch Observatory · GOV.UK Crime and Policing Act 2026 CSAM factsheet · Gibson Dunn on the EU AI Act Digital Omnibus · TechTimes on the Digital Omnibus nudifier ban