この記事は現在英語でのみご利用いただけます。
AI Incident Response: What to Do When Your AI System Fails or Causes Harm
AI systems fail differently from conventional software — systematic bias, model drift, hallucination. When they do, the response has legal, regulatory, and reputational dimensions that standard incident response playbooks do not address.
Key Takeaways
AI incidents can affect thousands simultaneously through systematic bias, be difficult to detect (gradual model drift), and have retrospective implications for decisions already made with flawed AI.
EU AI Act Article 73 requires providers and deployers of high-risk AI to notify national market surveillance authorities of serious incidents — within 15 days (general), within 10 days (death), within 2 days (widespread infringement or critical infrastructure disruption).
APRA CPS 230 requires notification of material operational risk incidents within 72 hours; disruptions to critical operations outside tolerance within 24 hours. GDPR requires DPA notification within 72 hours where AI incidents involve personal data breaches.
The first 24 hours: containment (limiting further harm), evidence preservation (logs, model versions), initial scope assessment (individuals affected, nature of harm), and notification decisions.
Retrospective remediation — addressing harm from decisions made before the failure was identified — requires planning before an incident occurs. Design AI systems to be remediable.
Post-incident, EU AI Act high-risk AI requires documented investigation, root cause analysis, corrective action, and updated conformity assessment. Regulators expect evidence that incidents produce systemic learning.
"情報提供のみを目的としています。この記事は法律、規制、財務または専門的なアドバイスを構成するものではありません。具体的なアドバイスについては、資格を持つ専門家にご相談ください。"
What makes AI incidents distinctive
Systematic bias produces unfair outcomes for a demographic group across thousands of decisions before anyone identifies the pattern. Hallucination produces confident but wrong information acted on before its inaccuracy is apparent. Model drift produces gradually degrading performance only visible through aggregate analysis. The scope of an AI incident is often not immediately apparent — harm may have been occurring for months before detection.
Regulatory notification obligations
EU AI Act Article 73: providers and deployers of high-risk AI must notify national market surveillance authorities of serious incidents — within 15 days (general), within 10 days (death), within 2 days (widespread infringement or critical infrastructure disruption). APRA CPS 230: material operational risk incidents require notification within 72 hours; critical operation disruptions outside tolerance within 24 hours. GDPR Articles 33-34: DPA notification within 72 hours and individual notification where high risk exists where AI incidents involve personal data breaches.
First 24 hours and retrospective remediation
Containment first — specific individuals should be pre-authorised to take production systems offline without extended approval chains. Evidence preservation simultaneously: preserve model versions, configurations, and relevant logs. Plan for retrospective remediation before incidents occur: maintain records of AI-informed decisions, preserve model versions, and keep logs enabling identification of affected individuals. EU AI Act high-risk AI requires documented investigation, root cause analysis, and updated conformity assessment after incidents.