Why governance gaps are invisible until they aren't
The most dangerous property of AI governance failures is that they are silent. Unlike a system outage or a data breach, inadequate AI governance produces no immediate alert. The risk accumulates, in biased models, in unmonitored drift, in liability that accrues without anyone noticing, until an incident makes it visible.
By then, the options are reactive rather than preventive. The following five signs are not theoretical risks. They are the most common governance failures found when organisations are assessed against established frameworks including ISO 42001, the NIST AI RMF, and the EU AI Act.
Sign 1: You cannot name who is accountable for each AI system
The accountability test is simple. For each AI system your organisation uses, answer this question: if this system caused harm to a customer, employee, or third party, who would be held responsible?
If the answer is "the vendor," "the IT team," "the business unit," or "we'd figure it out at the time" — your governance is inadequate.
Accountability is the foundational element of AI governance. Every other control depends on it. You cannot monitor what nobody owns. You cannot escalate to someone who hasn't been identified. You cannot audit a system whose governance chain leads nowhere.
The test: Name one person, not a team, not a title, who is accountable for each of your five most consequential AI systems. If you cannot, accountability is not assigned.
Sign 2: You have no AI system inventory
You cannot govern what you cannot see. Shadow AI, where employees use unapproved tools or use approved tools in unanticipated ways, is endemic across industries.
If your organisation does not maintain a documented register of AI systems in use, covering at minimum: tool name, purpose, data inputs, and named owner, your risk exposure is larger than you know.
An incomplete inventory is almost as dangerous as no inventory. If the IT team maintains a register of "approved" AI tools but the marketing department is running AI copy generation through a personal subscription, and the HR team is using an AI-enhanced ATS that nobody assessed at procurement, the gap in the register is where the governance risk lives.
The test: Ask three people in different departments to list the AI tools they use in their work. Compare the results to your official AI system register. The delta is your shadow AI exposure.
Sign 3: Your AI usage policy is absent, outdated, or unread
A policy that employees have not read is not a policy, it is a document. The distinction matters because the purpose of an AI usage policy is not documentation. It is to change employee behaviour, create enforceable standards, and provide a basis for accountability when those standards are breached.
Signs that your AI usage policy is inadequate:
- It does not name specific approved tools
- It does not specify what data may or may not be entered into AI tools
- It was published more than 12 months ago and not reviewed since
- Fewer than 70% of employees can describe its key rules unprompted
- It has never been enforced
The most common policy failure is excessive abstraction. Policies that tell employees to "use AI responsibly" without defining what responsible means in operational terms provide no governance value.
The test: Ask five employees to describe the main rules in your AI usage policy. If the answers are inconsistent, vague, or non-existent, the policy is not functioning.
Sign 4: AI decisions affecting individuals have no human review process
Any AI system that influences decisions about individual people, hiring, credit, insurance, performance assessment, content moderation, creates obligations that most organisations are not meeting.
The obligations are both regulatory and ethical. The EU AI Act requires human oversight for high-risk AI decisions. GDPR creates rights to explanation and challenge for automated individual decisions. Australian privacy law has equivalent provisions. US state AI laws are creating similar requirements for employment and credit applications.
Beyond compliance, the practical risk is significant. AI systems produce errors. They can produce biased errors, systematically disadvantaging certain groups while appearing to perform well overall. Without a human review process, these errors compound, affect real people, and create liability that accumulates invisibly.
The test: Identify the three AI systems most likely to affect individual outcomes in your organisation. Document the human review process for each. If the process does not exist in documented form, it does not exist in operational form either.
Sign 5: You have not reviewed AI system performance since deployment
AI models drift. A model that performed well at deployment in 2023 may perform poorly in 2026 because the world has changed and the model has not.
Drift is not a hypothetical risk. It is a structural property of machine learning systems. Models trained on historical data embed historical patterns. When those patterns change, economic conditions shift, demographic compositions change, language evolves, model performance degrades.
The governance obligation is ongoing monitoring: defined performance metrics, monitoring cadences, escalation thresholds, and a process for remediation when performance falls below acceptable levels.
Most organisations do not have this. They test models before deployment, find acceptable performance, and then treat the model as a static system until something goes wrong.
The test: For your highest-risk AI system, answer these questions: What is its current accuracy against a held-out test set? When was it last evaluated? What would trigger a retraining or decommissioning decision? If you cannot answer, post-deployment monitoring is not in place.
What to do if you recognise these signs
Recognition is the most important step. Organisations that cannot see their governance gaps cannot address them.
The remediation sequence follows the risks:
- Assign accountability before building any other control, without owners, nothing else works
- Build the inventory, you need to know what you're governing
- Update the policy, name tools, set data rules, enforce them
- Establish human review for high-stakes AI decisions
- Build monitoring processes for deployed models
Each of these is achievable. None of them requires a complete transformation program. A documented owner, a maintained register, a readable policy, a review cadence, these are the minimum viable governance structure that separates organisations that are managing AI risk from organisations that are accumulating it.
