Healthcare AI Governance: Frequently Asked Questions

Five federal agencies have active healthcare AI regulatory programs:

• FDA: Regulates AI as Software as a Medical Device (SaMD). Approximately 1,200 AI/ML devices cleared or authorized cumulatively; 295 in 2025 alone. Draft guidance on AI/ML-enabled device labeling issued January 2025. Average review time: 150 days. Approximately 97% of AI/ML devices enter via the 510(k) pathway.
• ONC: HTI-1 Final Rule (December 2023) requires transparency for predictive Decision Support Interventions in certified EHRs. USCDI v3 baseline effective January 1, 2026. Enforcement discretion for certain criteria extended to February 28, 2026.
• CMS: Scrutinizes AI in Medicare Advantage prior authorization and coverage determination. Completing Payment Year 2018–2024 RADV audit backlog using AI-driven analytics to detect fraudulent diagnosis code inflation.
• OCR: Enforces HIPAA requirements for PHI used in AI model training, including Minimum Necessary Standard and breach notification obligations for AI-related incidents.
• FTC: Monitors consumer-facing health AI for deceptive practices. Has issued AI guidance applicable to health apps and wellness platforms operating outside HIPAA coverage.

ONC HTI-1 (Health Data, Technology, and Interoperability: Certification Program Updates, Algorithm Transparency, and Information Sharing Final Rule) was published December 13, 2023. It establishes the first mandatory transparency requirements for predictive Decision Support Interventions (DSIs) in certified health IT.

Under Section (b)(11), certified EHR systems must provide clinical end-users with source attributes for each predictive DSI:

• The intervention's developer identity
• Training data demographics and exclusion criteria
• Known biases in the model
• Intended use parameters and contraindications

USCDI Version 3 became the baseline standard January 1, 2026. ONC extended enforcement discretion for certain criteria through February 28, 2026. Health IT vendors who have not configured source attribute documentation for their predictive DSI features are out of compliance with certified EHR requirements — and their health system customers inherit the compliance exposure.

CMS has not yet issued a single comprehensive AI prior authorization rule, but enforcement activity is accelerating through several mechanisms. CMS is completing Payment Year 2018–2024 RADV audits using AI-driven analytics — organizations that used AI-assisted coding that inflated diagnosis codes face audit exposure. Colorado HB 1139 (state law, but aligned with CMS direction) prohibits insurance coverage denials based solely on AI recommendations without human review. CMS Interoperability and Prior Authorization rules require documented decision criteria for coverage determinations — AI-generated recommendations must be traceable and auditable. The practical requirement: any AI used in coverage or prior authorization decisions must have documented governance, human oversight protocols, and audit trails that CMS examiners can review.

The state AI legislative landscape is active and accelerating. Key laws by state:

• Colorado: SB24-205 (AI Act, full enactment June 2026) requires algorithmic impact assessments and risk management policies for high-risk AI in healthcare decisions. HB 1139 prohibits insurance coverage denials based solely on AI recommendations.
• California: AB 489 (effective January 1, 2026) prohibits AI chatbots from presenting as licensed healthcare professionals. AB 316 (effective 2026) removes the AI-acted-autonomously liability defense — healthcare organizations bear full liability for AI decisions. AB 2013 requires disclosure of AI training data.
• Texas: SB 1188 (effective September 2025) requires licensed practitioner personal review of all AI-generated clinical content before decisions, mandates patient disclosure, and requires U.S.-based EHR storage effective January 1, 2026.
• Utah: SB 149/SB 226 and HB 452 establish AI disclosure requirements applicable to healthcare consumer interactions.
• Illinois: AI laws address employment and consumer-facing AI with healthcare implications.
• Nevada: Emerging AI disclosure legislation under development.

No single national standard exists. Multi-state healthcare organizations must track each jurisdiction independently. See our state-by-state regulatory tracker for current status.

Yes, if you operate in Europe, license software to European entities, or your AI systems affect people in the EU. The EU AI Act entered into force August 1, 2024, with mandatory compliance for high-risk AI systems required by August 2, 2026.

Medical AI devices classified under EU MDR Class IIa/IIb/III or IVDR Class A-D are categorized as high-risk AI with full compliance obligations: conformity assessments, technical documentation, human oversight measures, quality management systems, and registration in the EU AI database. U.S. health IT vendors selling to European hospital or health plan customers — even without a European office — face extraterritorial application if their AI outputs affect EU residents. The extraterritorial reach is comparable to GDPR: if you process data about EU residents, you are subject to the regulation regardless of where your servers are located.

Required AI governance documentation includes:

1. AI Policy and Governance Charter — establishes committee composition, decision rights, approval workflows, and escalation protocols for evaluating, approving, and monitoring all AI tools.
2. Intervention Risk Management (IRM) Records — comprehensive risk analyses for each algorithm covering validity, fairness, safety, and security (FAVES criteria).
3. Source Attribute Documentation (ONC HTI-1) — transparency logs for predictive DSIs: training data demographics, exclusion criteria, known biases, intended use parameters — accessible to clinical end-users.
4. Predetermined Change Control Plans (PCCP) — FDA-mandated protocols for autonomous algorithm updates. 10% of 2025 AI/ML device clearances included an authorized PCCP.
5. Algorithmic Bias and Equity Impact Assessments — statistical audits demonstrating algorithm performance across demographic subgroups.
6. Incident Response and Breach Notification Playbooks — AI-specific protocols compliant with HIPAA breach notification rules, covering hallucinations and data poisoning scenarios.
7. AI Vendor Business Associate Agreements — updated to specifically address PHI use in model training, subprocessor obligations, and AI-related breach notification.
8. Software Bill of Materials (SBOM) — component inventory for AI/ML systems in clinical workflows, required for cybersecurity risk management under NIST and HITRUST standards.

A Predetermined Change Control Plan is an FDA-approved protocol that specifies in advance how an AI/ML-enabled medical device may autonomously update its algorithm without requiring a new 510(k) or PMA submission for each update. The PCCP describes: the types of modifications planned (performance improvements, dataset expansions), the methods used to implement and validate changes, and the controls to ensure modifications remain within safe and effective parameters.

FDA finalized PCCP guidance in 2025. Of 295 AI/ML devices cleared in 2025, 30 (approximately 10%) included an authorized PCCP. Organizations deploying devices with adaptive algorithms — those that retrain on new clinical data post-deployment — without an authorized PCCP may be making unapproved modifications to an FDA-cleared device, which is a significant regulatory violation. Applying for a PCCP as part of the initial 510(k) submission is the correct approach for any adaptive algorithm.

OCR has signaled that training AI models on PHI implicates several HIPAA requirements:

• Minimum Necessary Standard: Covered entities must limit PHI used for AI training to the minimum necessary to accomplish the purpose — bulk PHI exports for training purposes may violate this standard.
• Business Associate Agreements: Must specifically address what PHI AI vendors may use for training, how it must be de-identified or limited, subprocessor obligations, and breach notification protocols specific to AI training environments.
• De-identification requirements: The Privacy Rule's Safe Harbor and Expert Determination methods govern whether AI training datasets qualify as de-identified. "Anonymized" training data that can be re-identified through model outputs is not compliant de-identification.
• Research exception: PHI used for AI development may qualify for the research exception under certain conditions — but this requires an IRB approval or waiver process, not just an internal determination.

A functional AI governance committee requires four core roles using the PPTO framework: (1) Technical AI Lead / Chief Medical Information Officer (CMIO) — currently present on 45% of AI committees; provides technical evaluation of algorithm validity and performance. (2) Compliance and Privacy Manager — HIPAA, NIST AI RMF, and state law expertise; responsible for regulatory mapping and documentation. (3) Clinical AI Specialist — monitors algorithm accuracy, hallucinations, and real-world drift post-deployment. (4) AI Ethics Officer — equity impact assessments; currently present on only 25% of committees despite the 75% gap this creates in bias and equity oversight.

Committee effectiveness requires documented authority: approval is required before any AI tool enters clinical or coverage workflows, and the committee has standing to reject or suspend tools. Without formal approval authority, the committee is advisory only and does not satisfy regulatory pre-implementation approval requirements.

Project-based AI governance program builds range from $75,000 to $250,000+ depending on organizational size, complexity, and regulatory scope. Managed service retainers for ongoing governance maintenance run $50,000 to $180,000+/month at enterprise scale. Big 4 (Accenture, Deloitte) enterprise AI transformation retainers exceed $2,000,000+. IHS serves mid-market healthcare organizations with project-based engagements without Big 4 overhead.

The cost of non-compliance provides context: FDA PMA user fees for AI-enabled medical devices are $531,163 (FY2026 standard fee, reduced to $113,750 for small businesses). HITRUST r2 assessor fees run $60,000–$150,000+. CMS RADV audit penalties for fraudulent coding can be substantial. ONC-ATL certification fees for health IT vendors: $15,000–$30,000 annually for Single Patient API, $7,000–$28,000 for Bulk Data API. A proactive governance program is materially less expensive than reactive remediation after an adverse finding.

URAC and ACHC accreditation standards require quality management, health equity, and utilization management infrastructure that directly overlaps with AI governance requirements. For URAC-accredited health plans: health equity and quality standards already require documented processes for identifying and mitigating disparate outcomes — algorithmic bias auditing extends that existing framework to AI-assisted coverage decisions without creating a separate compliance effort. For URAC-accredited specialty pharmacies: AI-assisted dispensing workflow governance fits within existing clinical quality oversight requirements. IHS positions AI governance as an extension of your existing accreditation program, reducing total compliance cost and leveraging documentation infrastructure already in place.