AI Governance for US Healthcare Organisations: FDA, HIPAA, CMS, and State Requirements

Healthcare AI in the US is regulated by FDA as a medical device, subject to HIPAA for data handling, and faces increasing CMS oversight for AI in Medicare and Medicaid decision-making. Here is the governance framework.

Key Takeaways

AI used for diagnosis, treatment recommendation, risk prediction, or patient monitoring is regulated by FDA as Software as a Medical Device (SaMD). FDA clearance (510(k)), De Novo authorisation, or PMA approval is required before clinical deployment.
HIPAA's Privacy and Security Rules apply to all AI systems that create, receive, maintain, or transmit protected health information. Business Associate Agreements must be in place with AI vendors before any PHI is processed.
CMS issued guidance in 2024 requiring Medicare Advantage plans to ensure AI-driven prior authorisation decisions are based on individual patient circumstances — not population-level statistical models. Plans whose AI systematically overrides individual clinical presentations face enforcement.
ONC's HTI-1 rule includes transparency requirements for AI algorithms in certified health IT — algorithms must be disclosed to clinicians who use AI-assisted decision support tools.
Clinical validation — demonstrating AI tool performance in your specific patient population — is a regulatory and liability requirement. Vendor validation studies on different populations may be insufficient.
Healthcare organisations face clinical negligence liability for AI-assisted decisions that cause patient harm. Clinical staff must be trained on AI tools' limitations — reliance on AI without understanding its capabilities does not satisfy the professional standard of care.

"情報提供のみを目的としています。この記事は法律、規制、財務または専門的なアドバイスを構成するものではありません。具体的なアドバイスについては、資格を持つ専門家にご相談ください。"

FDA regulation of clinical AI

The FDA regulates Software as a Medical Device (SaMD) through its Center for Devices and Radiological Health. AI intended for diagnosis, cure, mitigation, treatment, or prevention of disease is generally regulated as a medical device. Most AI cleared through the 510(k) pathway or De Novo pathway for diagnostic AI in radiology, ophthalmology, cardiology, and pathology. Healthcare organisations must verify that clinical AI has appropriate FDA clearance before deployment — using uncleared AI in clinical settings creates significant legal exposure. Check the FDA's 510(k) and De Novo databases at accessdata.fda.gov.

HIPAA and AI

HIPAA applies to all AI tools that create, receive, maintain, or transmit protected health information (PHI). The most common governance failure: using AI tools without a Business Associate Agreement. Most general-purpose AI tools are not HIPAA compliant without specific BAA and configuration. Verify that every AI tool touching PHI has a signed BAA and meets HIPAA Security Rule requirements — encryption, access controls, audit logging, and breach notification procedures.

CMS and Medicare Advantage AI

CMS guidance (2024) clarified that Medicare Advantage plans must ensure utilisation management decisions informed by AI are based on individual patient circumstances rather than population-level statistical averages. This was a direct response to Senate investigations finding some MA plans used AI to systematically deny claims. Plans with unusually high AI-assisted denial rates should expect heightened CMS scrutiny. Claims review must involve a qualified clinician who genuinely considers the individual patient's specific clinical situation.

Clinical responsibility

GMC (and equivalent state medical boards) guidance is consistent: clinicians retain professional responsibility for decisions made with AI assistance. Before deploying AI in a clinical pathway, ensure: clinicians have received adequate training on the tool's capabilities, limitations, and failure modes; a clinical lead is designated for oversight; ongoing performance monitoring is in place; and there is a clear process for reporting AI-related clinical concerns and for reviewing AI performance changes.

英語で読む