AI Agents and GRC: The 2026 Guide to Governance, Risk, and Compliance for Autonomous AI

What an AI agent actually is — and what most products labelled as agents are not

An AI agent, in the precise technical sense relevant to governance and risk, is an AI system that perceives an environment, reasons about goals, plans actions, executes those actions through external tools or systems, and adapts its behaviour based on outcomes — with limited or no human oversight in the loop. The defining characteristic is autonomous goal pursuit. An agent can decide what to do next without being explicitly told.

This is materially different from three adjacent technologies that are often conflated with AI agents in commercial marketing:

Generative AI assistants respond to prompts. A chatbot or copilot generates content in response to a request and then stops. It does not maintain persistent goals across interactions. It does not take autonomous actions in external systems. It augments a human worker; it does not replace human decision-making.

Robotic Process Automation (RPA) follows scripted rules. An RPA bot executes a deterministic sequence of steps — open this application, copy this field, paste it there, submit. RPA is automated but not adaptive. When the input changes in an unexpected way, RPA bots typically fail rather than reason about alternatives.

Multi-agent systems are a different and more advanced category. An AI agent is typically a single model equipped with tools to perform end-to-end tasks. Agentic AI as a system architecture is often composed of multiple agents that coordinate, delegate sub-tasks, and pursue broader composite goals. The University of California Berkeley Center for Long-Term Cybersecurity makes this distinction explicit in its February 2026 Agentic AI Risk-Management Standards Profile, and the distinction matters for governance — single-agent oversight is qualitatively different from multi-agent oversight.

The market reality is that this technical clarity is rare. Gartner has formally identified the practice of "agent washing" — the rebranding of existing chatbots, AI assistants, and RPA tools as agentic AI without substantial agentic capabilities — and estimates that only about 130 of the thousands of agentic AI vendors offer genuine agentic capability. The first practical governance step for most organisations is not building an AI agent strategy. It is determining which of the systems already deployed inside the organisation, under the "AI agent" label, actually meet the technical definition.

Adoption is real, but it is outpacing governance by a large margin

Several primary-source data points converge on the same conclusion: AI agents are moving into production at enterprise scale, and governance is not keeping pace.

Gartner's June 2025 press release, drawing on a January 2025 poll of 3,412 webinar attendees, found that 19% of organisations had made significant investments in agentic AI, 42% had made conservative investments, 8% had made no investment, and 31% were taking a wait-and-see approach or were unsure. By Gartner's own forward forecast, at least 15% of day-to-day work decisions will be made autonomously through agentic AI by 2028 (up from 0% in 2024), and 33% of enterprise software applications will include agentic AI by 2028 (up from less than 1% in 2024).

Gartner's same press release issued its most quoted prediction: over 40% of agentic AI projects will be cancelled by the end of 2027 due to escalating costs, unclear business value, or inadequate risk controls. Senior Director Analyst Anushree Verma stated: "Most agentic AI projects right now are early stage experiments or proof of concepts that are mostly driven by hype and are often misapplied. This can blind organisations to the real cost and complexity of deploying AI agents at scale, stalling projects from moving into production."

The governance gap is not theoretical. Deloitte's State of AI in the Enterprise 2026 report found that only one in five companies has a mature model for the governance of autonomous AI agents. The lag is structural: organisations are deploying agents into production environments faster than they are building the inventory, monitoring, identity, and accountability infrastructure needed to govern them.

Why AI agent risk is structurally different from other AI risk

Most enterprise AI governance frameworks — including the foundational NIST AI Risk Management Framework (AI RMF) and its July 2024 Generative AI Profile (NIST AI 600-1) — were designed for a world in which AI systems make predictions or generate content for human review. AI agents change the risk topology in five specific ways that compliance, risk, and audit functions need to understand.

Autonomous action with external effect. A predictive model that recommends a credit decision is a different risk category from an agent that approves the credit, transmits the decision to the customer, updates the core banking system, and schedules the funds disbursement. The first requires bias and accuracy controls. The second requires those controls plus rollback procedures, action-level authorisation, audit logging of every autonomous decision, and a defined process for stopping or reversing actions in flight.

The University of California Berkeley Center for Long-Term Cybersecurity, in its Agentic AI Risk-Management Standards Profile published February 2026, identifies this category of risk as including "unintended goal pursuit, unauthorised privilege escalation or resource acquisition, and other behaviours — such as self-replication or resistance to shutdown — that could result in systemic or catastrophic harm." The framing is deliberately strong because the failure modes are qualitatively different from those of static models.

Identity and access management built for humans. Enterprise identity systems — Single Sign-On, role-based access control, privileged access management — were designed around the assumption that the actor requesting access is a human authenticating with credentials they alone possess. AI agents break this assumption. An agent acting on behalf of a user, with that user's credentials, can take actions the user did not authorise. An agent with system-level service credentials has effective superuser access without any of the human controls (approval workflows, justification, time-bound elevation) that human privileged access requires.

The Australian Prudential Regulation Authority (APRA) named this gap directly in its 30 April 2026 letter to industry on artificial intelligence: "identity and access management capabilities have not yet adjusted to non-human actors such as AI agents." APRA observed this finding from its targeted supervisory engagement with selected large Australian banks, insurers, and superannuation trustees in late 2025. The fact that Australia's prudential regulator chose to publish this observation in plain language signals that it is now testable in supervisory reviews.

Cascading failures across delegation chains. Multi-agent systems amplify errors. An agent that miscalculates a parameter and passes the bad input to a downstream agent can trigger a cascade of compounding errors before any human is positioned to intervene. The time horizon for human oversight collapses in agentic systems: failures can propagate in milliseconds, while the human-in-the-loop control architecture in most enterprises assumes minutes to hours.

Probabilistic and non-deterministic behaviour. Conventional change-management and assurance processes assume a stable artefact. Once a system is tested and released, it behaves the same way until the next release. AI agents do not. The same agent, given the same input, may produce different outputs across runs. Model drift, context-dependent reasoning, and the influence of system prompts all mean that point-in-time assurance is insufficient. APRA's April 2026 letter is specific on this point: it observed "reliance on point in time and sample based assurance methods, despite these methods being ill suited to probabilistic models that learn, adapt and degrade over time."

Novel attack surfaces. AI agents introduce attack pathways that conventional information security controls were not designed to detect. APRA's letter explicitly named: prompt injection, data leakage, insecure integrations, exploit injection, and the manipulation or misuse of autonomous AI agents. A web application firewall does not catch prompt injection. A conventional penetration test does not assess an agent's tool-use attack surface. The security testing programme that protected the environment in 2024 is structurally inadequate for the agent-augmented environment of 2026.

The global regulatory landscape on AI agents specifically

No major jurisdiction has yet enacted AI-agent-specific legislation. Regulators are instead applying existing frameworks — AI legislation, sectoral prudential standards, privacy law, consumer protection law — to agentic systems, while developing supplementary guidance.

European Union — EU AI Act. The EU AI Act does not contain an "AI agent" definition or a separate regulatory category for agents. AI systems that meet the AI Act's definition of an AI system, and that fall within Annex III (employment, credit, biometric, education, law enforcement, critical infrastructure, essential services) or are embedded in regulated products under Annex I, are subject to the high-risk obligations regardless of whether they are agentic. Following the Digital Omnibus on AI provisionally agreed on 7 May 2026, the Annex III high-risk obligations apply from 2 December 2027 and the Annex I embedded-systems obligations from 2 August 2028. Transparency obligations under Article 50 — including the requirement to inform users they are interacting with an AI system — apply from 2 August 2026 and were not extended by the Omnibus. For consumer-facing AI agents in the EU, the Article 50 transparency obligation is the most immediate compliance trigger.

The General-Purpose AI (GPAI) model obligations in Articles 50 to 55 of the EU AI Act apply to upstream model providers, not to agent deployers. However, organisations building agents on third-party foundation models should be aware that the systemic-risk classification of the underlying model — and the model provider's obligations around evaluation, adversarial testing, and serious incident reporting — affect downstream agent risk.

United States — NIST. The United States National Institute of Standards and Technology (NIST), through its Center for AI Standards and Innovation (CAISI), published a Request for Information on the security of AI agent systems in the Federal Register on 8 January 2026, with comments closing on 9 March 2026. The RFI scoped AI agent systems as those "capable of taking actions that affect external state, i.e., persistent changes outside of the AI agent system itself" and explicitly noted that retrieval-augmented generation systems not orchestrated to act autonomously, and chatbots, fall outside its scope. The RFI is the most authoritative US Government statement to date on what constitutes an AI agent for governance purposes.

The NIST AI RMF Generative AI Profile (NIST AI 600-1), released 26 July 2024, addresses generative AI risks but does not directly address agentic autonomy. Sector-specific profiles are emerging — NIST released a concept note for an AI RMF Profile on Trustworthy AI in Critical Infrastructure on 7 April 2026. Practitioners working ahead of NIST's official profile have produced the Berkeley Center for Long-Term Cybersecurity Agentic AI Risk-Management Standards Profile (February 2026) and the Cloud Security Alliance Agentic AI NIST AI RMF Profile, both of which apply the NIST AI RMF four functions (Govern, Map, Measure, Manage) to agentic systems.

OECD. The OECD AI Policy Observatory tracks national AI policies and maintains the Catalogue of Tools and Metrics for Trustworthy AI. The OECD AI Principles, updated in 2024, apply to all AI systems but do not yet contain agent-specific provisions.

Australia: the most developed practical framework for AI agent governance

Australia has not enacted AI-specific legislation comparable to the EU AI Act. The Australian Government's approach, articulated in the National AI Plan released on 2 December 2025, is to work through existing technology-neutral law — privacy law, consumer law, work health and safety law, anti-discrimination law, sectoral prudential standards — supplemented by guidance and voluntary standards. This is sometimes characterised as a weakness; it is more accurately characterised as a deliberate choice that places significant operational guidance ahead of legislation.

APRA Letter to Industry on AI — 30 April 2026. Issued by APRA Executive Board Member Therese McCarthy Hockey, this letter is the most consequential AI-specific regulatory communication for Australian financial services in 2026. Drawing on a targeted supervisory engagement with selected large banks, insurers, and superannuation trustees in late 2025, APRA set out specific expectations across four observation areas: governance, cyber and information security, supplier risk, and change management and assurance.

Key APRA observations and expectations on agents:

• Boards "are still developing the technical literacy required to provide effective challenge on AI-related risks and oversight," with "overreliance on vendor presentations and summaries." APRA's stated minimum expectation: boards must "maintain sufficient understanding and literacy with respect to AI in order to set strategic direction and provide effective challenge and oversight."
• Required governance arrangements include, at a minimum: frameworks (policy, standard, guidance) and reporting lines; ownership and accountability across the AI lifecycle; an inventory of AI tooling and use cases; human involvement for high-risk decisions and accountability; and training and education of staff.
• Specific cyber and information security expectations: strong privileged access management; timely patching; hardened configurations; automated vulnerability discovery; penetration testing; and "controls over agentic and autonomous workflows."
• Supplier risk: mapping and maintaining visibility over the full AI supply chain, including material third-party and fourth-party dependencies. Concentration risk is flagged where entities depend heavily on a single provider for multiple AI use cases.
• Change management and assurance: continuous validation; integrated assurance across cyber, data governance, model risk, operational resilience, privacy, and conduct; and second-line risk and internal audit functions with the technical capability "to independently assess AI systems including probabilistic models and agentic workflows."

APRA's regulatory hook is the existing prudential framework: CPS 220 (Risk Management), CPS 230 (Operational Risk Management, including operational resilience), CPS 234 (Information Security), and CPS 510 (Governance). APRA has not announced new prudential standards. The April 2026 letter is the signal that AI governance will now be tested against these existing standards in entity prudential reviews and thematic activities.

ASIC Letter — 8 May 2026. The Australian Securities and Investments Commission published an open letter on 8 May 2026 that complements APRA's, with broader application beyond APRA-regulated entities. ASIC expects boards and senior executives "to be able to understand their organisation's position, ask the right questions and be satisfied that their cyber resilience measures are proportionate to their organisation and threat environment." This builds on ASIC's October 2024 REP 798 "Beware the Gap" report on AI governance arrangements, which identified a governance gap across financial services even before agentic AI became a material concern.

DTA Policy for the responsible use of AI in government. The Digital Transformation Agency's Policy for the responsible use of AI in government became effective on 1 September 2024 and has since been updated to version 2.0. The DTA has built out the most comprehensive operational AI governance suite of any English-speaking national government, including:

• AI Plan for the Australian Public Service 2025, setting the APS roadmap for AI adoption.
• Technical standard for government's use of artificial intelligence, setting technical requirements for AI systems across their full lifecycle from initial design through monitoring and decommissioning.
• AI Impact Assessment Tool with supporting guidance, to help agencies identify, assess, and manage AI use case impacts and risks against Australia's AI Ethics Principles.
• Guidance on AI procurement in government.
• Pilot AI Assurance Framework (piloted from September 2024), establishing the AI assurance approach for APS use cases.
• AI Review Committee, to advise on high-risk AI use cases across the APS — terms of reference being finalised, with further details signalled for Q1 2026.

The DTA framework is not law and does not apply outside the APS. However, the AI Impact Assessment Tool and the Technical Standard for AI are referenced and used by private-sector organisations as practical implementation references, and the DTA's approach is consistent with the Voluntary AI Safety Standard published by the Department of Industry, Science and Resources.

National Framework for the Assurance of AI in Government. The Data and Digital Ministers Meeting agreed and released the National Framework for the Assurance of AI in Government on 21 June 2024. It establishes cornerstones and practices for governments at federal, state, and territory level applying Australia's AI Ethics Principles to their AI assurance.

Privacy Act amendments — automated decision-making transparency. The Privacy and Other Legislation Amendment Act 2024 introduced new automated decision-making (ADM) transparency obligations, which take effect on 10 December 2026. From that date, APP entities will be required to include in their Privacy Policies information about the kinds of personal information used in substantially automated decisions that have a legal or similarly significant effect on individuals. AI agents that make or substantially contribute to such decisions are squarely within scope. The Office of the Australian Information Commissioner (OAIC) is expected to publish detailed guidance ahead of the December 2026 commencement.

Practical implementation: how to operationalise AI agent GRC

The gap between the regulatory expectation and the operational reality is substantial. Most organisations need to do six things, in approximate priority order.

1. Build an AI agent inventory and classification. Every AI agent deployed in production — including embedded agents in SaaS tools, copilots in productivity suites, and shadow IT — should be inventoried with: owner (named individual), purpose, data accessed, systems acted upon, autonomy level, business criticality, and risk classification. This is the precondition for everything else. APRA explicitly identified the absence of AI inventories as a finding in its April 2026 letter.

2. Classify agents by autonomy level. Not all agents require the same controls. A useful classification is: (a) advisory agents that recommend actions for human approval; (b) tactical agents that execute pre-approved actions within tightly scoped parameters; (c) operational agents that pursue defined goals across multi-step workflows with periodic human checkpoints; (d) autonomous agents that operate continuously with minimal human oversight. The classification determines the appropriate identity model, human-in-the-loop controls, monitoring intensity, and rollback architecture.

3. Treat agents as non-human identities. Each agent should have its own identity, distinct from any human user's identity. That identity should be subject to: scoped permissions (least privilege), time-bound credentials, logged actions, separation between agent identity and human identity for audit purposes, and revocation capability. Agents acting under a user's credentials should be exceptional, not standard, and the user should be informed and consent.

4. Build a continuous assurance layer. Point-in-time testing is insufficient. The assurance layer for agents needs continuous validation: behaviour monitoring, drift detection, output sampling and review, anomaly detection on actions taken, and integration with the security operations centre. This is what APRA means by "continuous validation or monitoring in place to detect issues such as model drift, bias, failure modes, or control breakdowns in a timely manner."

5. Define rollback and kill-switch architecture. Every agent in production should have a documented, tested process for: stopping the agent in flight; reversing or compensating for actions taken; isolating the agent from systems it has accessed; and re-establishing the pre-agent state. The Writer 2026 enterprise AI survey found that 35% of executives admitted they could not immediately "pull the plug" on a rogue AI agent. This is a fundamental control gap.

6. Update board governance. AI agent risk is a board-level matter in any organisation where agents are deployed in customer-facing, regulated, or high-impact processes. Board-level controls include: a clearly designated executive accountable for AI agent risk (typically the Chief Risk Officer, Chief Technology Officer, or Chief Information Security Officer, depending on the organisation); regular reporting on the AI agent inventory and incident pipeline; AI literacy training for directors that is independent of vendors; and explicit consideration of AI agent concentration risk in third-party risk reviews.

Strategic impact: what AI agents change about the organisation

Beyond the immediate compliance work, AI agents have second-order strategic effects that are easier to identify in 2026 than to plan for. Three are worth flagging.

First, the unit of competition shifts from process to outcome. When an agent can perform a multi-step workflow autonomously, the organisation's competitive position depends less on whether it has the workflow documented and more on whether the outcome is achieved. This places a premium on outcome measurement and on the data infrastructure that allows outcomes to be attributed to specific actions.

Second, organisational design changes. Layers of middle management whose primary function was orchestration — task allocation, hand-offs, status reporting, exception escalation — become structurally redundant where agents perform those functions. This is not the same claim as "AI will replace knowledge workers," which is contested and largely speculative. It is a narrower and more defensible claim: organisations that deploy agents successfully will operate with flatter coordination structures because coordination cost falls.

Third, supplier concentration risk becomes systemic. As Microsoft, Google, OpenAI, Anthropic, and a small number of others embed agentic capabilities deep in productivity tooling, the dependency picture for individual organisations resembles cloud dependency a decade ago — but with materially higher switching cost. APRA's letter explicitly flagged this: "some entities heavily dependent on a single provider for multiple AI use cases." Documented contingency planning, exit strategies, and substitution testing are no longer optional.

What to do next

For organisations starting from a low base, three actions are achievable in the next 90 days and will materially improve the governance position.

One: complete the AI agent inventory. Catalogue every system in production that meets the agent definition, including embedded SaaS agents. Assign an accountable owner for each. This single artefact is what APRA, ASIC, and most other regulators are now asking for.

Two: classify the inventory by autonomy level and business criticality, and identify the agents that operate above the "human approval per action" threshold. These are the systems that require the most attention.

Three: assess the kill-switch and rollback capability for the highest-criticality agents. If the answer to "can we stop this agent immediately and reverse what it has done?" is not a confident yes, that is the first remediation priority.

For organisations further along, the priority shifts to building the continuous assurance layer and aligning the audit, risk, and security functions to test agentic behaviour rather than static models. This is structurally harder than the inventory work and requires investment in tooling, skills, and process redesign. The organisations that succeed will be those that treat AI agent governance as infrastructure rather than overhead — embedded in architecture from the start, with clear ownership and measurable outcomes.

For Australian financial services entities specifically, APRA's April 2026 letter is the operative document. Reading it carefully — including the worked expectations for each of the four observation areas — and conducting a structured gap assessment against those expectations is the right immediate priority. APRA has signalled that proportionate prudential reviews, thematic activities, and AI supplier engagement will form an active supervisory programme over the next 12 months. The window for structured early engagement with the APRA Non-Financial Risk Team — which the letter expressly invites — is open now, and will produce a materially softer supervisory posture than waiting to be examined.