Most conversations about AI regulation in Australia reach for privacy law, consumer law, or the copyright debate. The online safety framework is often left out, which is a mistake for anyone whose product can generate or move content. The Online Safety Act 2021 (Cth), administered by the eSafety Commissioner, gives Australia's online safety regulator powers over harmful online content and conduct, and generative AI now sits squarely inside that frame.
The reason is simple. The Act was written around what content does and how services handle it, not around any particular technology. When a system can produce a deepfake, synthesise an intimate image of a real person, or generate abuse material, the harm is the same regardless of whether a human or a model created it. This explainer sets out what the Act requires, where AI comes in, and what a risk or compliance operator should build.
What the Online Safety Act requires
The Act is not a single obligation but a set of connected mechanisms that the eSafety Commissioner can bring to bear on online services. Understanding the shape of the framework matters more than memorising any one provision, because the pieces work together.
The Basic Online Safety Expectations
The Basic Online Safety Expectations are a set of expectations that apply to online services about how they keep users safe. They are framed around the idea that services should take reasonable steps to prevent and address harmful material and conduct, and the Commissioner can ask services to report on how they are meeting those expectations. The Expectations are deliberately broad, which is what allows them to stretch across new service types as technology moves.
Industry codes and standards
The Act supports industry codes and standards that set out how categories of online services must deal with certain classes of harmful material. These codes and standards have been developed to cover categories of online services, and that category-based structure has been extended to cover generative AI services. The practical effect is that a service does not escape the framework simply by being new. If it falls within a covered category, the relevant code or standard can apply to it.
Removal and reporting schemes
The Act establishes removal and reporting schemes that let people report harmful material and, in defined circumstances, seek its removal. These include schemes directed at image-based abuse, meaning intimate images shared without consent, and at seriously harmful content. The Commissioner can act on reports and, where the framework allows, require material to be taken down. For an operator, this is the sharp edge: a removal notice is an operational event that demands a fast, documented response.
Where AI comes in
eSafety has publicly focused on the risks posed by generative AI, and the pathways from AI to captured harm are concrete rather than hypothetical. Three stand out.
Deepfakes and synthetic media
Deepfakes are AI-generated or AI-altered media that depict a real person doing or saying something they did not. Where such material is harmful, it can engage the online safety framework in the same way other harmful content does. A service that generates convincing synthetic depictions of real people is producing exactly the kind of material the framework is concerned with.
AI-generated image-based abuse
AI-generated image-based abuse is a fast-growing concern. Generative tools can now synthesise intimate images of real people without their consent, and the resulting material falls within the same image-based abuse concerns the Act addresses. The person depicted never posed for anything, but the harm and the reporting pathway are real. A business hosting or offering image-generation capability should treat this as a foreseeable misuse, not an edge case.
AI-generated child sexual abuse material
The most serious pathway is AI-generated child sexual abuse material. The capacity of generative models to produce such material is a central reason eSafety has pressed on generative AI risk. For any service with generative capability, preventing this outcome is not a compliance nicety, it is a baseline expectation, and it is the area where regulator and public attention is most intense.
Why your AI service may be captured
The common thread is that businesses deploying or hosting generative AI that can produce or distribute harmful content may fall within these expectations and codes. Being an AI vendor rather than a social media platform does not put you outside the frame. If your service can generate or spread captured content, the Basic Online Safety Expectations, the relevant codes and standards, and the reporting and removal schemes may all be relevant to you. eSafety expects services to have safety-by-design measures, which means safety built in before harm occurs rather than added after a complaint.
What to do
The framework rewards services that have thought about harm in advance and can show it. The following steps translate the expectations into an operating posture.
Assess whether you are captured. Map your AI product against the online safety framework. Can it generate or distribute the classes of harmful material the Act addresses, including deepfakes, non-consensual intimate imagery, or abuse material? If yes, assume the Basic Online Safety Expectations and the relevant codes and standards are in scope, and document that assessment.
Adopt safety-by-design. Build safety measures into the product rather than bolting them on. That includes input and output filtering, guardrails against generating depictions of real people without consent, and technical controls aimed specifically at preventing the most serious material. Safety-by-design is the expectation eSafety has stated most clearly, so make it demonstrable.
Stand up reporting and takedown processes. Give users a clear, usable way to report harmful content, and build an internal path to triage and act on those reports quickly. Removal notices and user reports are operational events, so define who owns them, what the response time is, and how each step is recorded.
Keep evidence. Record your design decisions, your safety controls, your reporting flows, and how you respond to notices. If the Commissioner asks how you meet the Expectations, the answer should be a documented process, not an assurance.
Watch the codes as they evolve. The category-based structure means codes and standards can extend to new service types, including generative AI. Treat the applicable instruments as living obligations and reassess when your product or the codes change.
Frame the regulator accurately. The eSafety Commissioner administers and enforces the online safety framework. It is not a general AI regulator, and it does not license models. Its leverage is over harmful content and conduct, and over how services handle them, which is precisely where a generative AI product creates exposure.
If you are not sure whether your AI service sits inside the online safety framework, that uncertainty is itself the finding worth acting on. A short, structured review of where your product touches harmful-content pathways will tell you far more than a general assurance that you are low risk. The free AIRA Health Check is a practical starting point for that review, helping you locate where your AI deployment intersects the Online Safety Act and where safety-by-design and reporting processes still need to be built.