What a Hospital's AI Governance Program Is Supposed to Do

A hospital's AI governance program exists to keep clinical artificial intelligence safe once it is in daily use, long after the sales demo is over. In practice that means a named group inside the organization that decides which tools are permitted, confirms each one works on the local patient population, watches for performance drift, screens for bias and risk, guards patient data, tells patients when AI touches their care, trains the people who depend on it, and reports safety problems so other hospitals can learn. In September 2025 the Joint Commission and the Coalition for Health AI (CHAI) published the first shared baseline for what such a program should contain.

Key points#

• AI governance is about what happens after a tool is bought, not whether the vendor's pitch sounded good.
• The September 2025 Joint Commission and CHAI guidance names the parts a credible program needs, from a real oversight body to confidential safety reporting.
• A voluntary Joint Commission certification followed on June 1, 2026, recognizing organizations that can show these safeguards are actually running.
• The certification recognizes how a hospital uses AI; it does not vouch for individual AI products.
• Anyone, including patients, can test a program with a few plain questions about who monitors a tool and what happens when it drifts.

Why this guidance appeared when it did#

Clinical AI moved faster than the systems meant to supervise it. Tools that draft notes, flag deteriorating patients, read images, and triage messages arrived in hospitals before most organizations had any settled way to decide which ones to trust. The September 17, 2025 guidance, titled Responsible Use of AI in Healthcare, was framed by both groups as a first-of-its-kind attempt to give health systems a common playbook.

The two authors bring different strengths. The Joint Commission accredits most American hospitals, so its attention to a topic usually signals where oversight is heading. CHAI is a nonprofit coalition of nearly 3,000 member organizations, including health systems, patient advocacy groups, technology firms, and startups, assembled to hammer out consensus rules for trustworthy health AI. Rather than treat AI as an exotic new category needing its own bureaucracy, they placed it inside the governance and compliance structures hospitals already run.

The parts a credible program has to include#

The guidance describes a set of foundational elements. Translated out of policy language, they are the working parts every serious program needs.

A real oversight body with written rules. A standing committee with genuine authority and mixed membership, drawing on compliance, information technology, clinical leadership, operations, and data privacy. One enthusiastic champion in a single department is not governance. The written policies should state plainly what AI may and may not be used for.

Validation before deployment. Before a tool goes live, someone should confirm it performs on the organization's own patients, not only on the vendor's original dataset. Accuracy does not transfer automatically from one population to another.

Ongoing monitoring for drift. This is the most important element and the most often skipped. A model that looked excellent at launch can decay as patient mix, documentation habits, or upstream software change. The guidance expects continuous watching, not a one-time sign-off.

Risk and bias review. The organization should document each tool's risks and check whether it was tested on a population resembling its own. A model trained mainly on one demographic can be reliably wrong for the patients actually being seen.

Patient privacy and transparency. Clear rules on how patient data is accessed and used, and a willingness to tell patients when AI plays a part in their care.

Data security protections. Every use of patient data with an AI tool must meet HIPAA, backed by measures such as encryption, restricted access, and written agreements defining permitted uses.

Education for the people who use it. Staff should be taught not just how to operate a tool but where it fails and what its limits are.

Confidential safety reporting. The guidance encourages blinded reporting of AI-related safety events to independent bodies, so a failure in one hospital can warn the rest of the field. It borrows the logic that made aviation and medication-error reporting effective.

From voluntary guidance to a certification#

The 2025 guidance carried no accreditation weight; it was a shared reference, nothing more. That began to change. On June 1, 2026, the Joint Commission launched a voluntary Responsible Use of AI in Healthcare certification for organizations that can demonstrate governance, safeguards, monitoring, and education are genuinely in place. Its standards cluster around governance, data management, reducing risk and bias, monitoring and validating safety and performance, and transparency with education and training.

Two limits are worth keeping in mind. The certification recognizes how an organization uses AI; it does not validate or certify any individual AI product. And an organization does not need to be Joint Commission accredited to apply. A voluntary certification is a signal rather than a mandate, and it is reasonable to expect its themes to migrate into future accreditation expectations over time.

Questions that reveal whether a program is real#

Endorsing a standard is easy; living it is not. A handful of questions separate a functioning program from a binder on a shelf.

• Who owns AI oversight, and has that body ever declined a tool? A committee that has approved everything it reviewed is probably a rubber stamp.
• Was the tool validated on our own patients, or only on the vendor's data?
• After go-live, is anyone measuring whether it still performs as claimed, and how would they detect drift?
• Can frontline staff describe the tool's known failure modes? Training that leaves no such knowledge did not happen.
• Does the organization report safety events externally, or only file them internally?
• Are patients ever told when AI is involved in their care?

None of these require technical fluency. They test whether the listed elements show up as behavior rather than paperwork. The pattern worth watching for is a confident claim of being AI-powered or AI-governed that thins out the instant you ask who monitors the tool and what they would do if it started failing. The discipline is the same one that underlies careful evidence appraisal: a claim has to match verifiable evidence, and the burden of proof sits with whoever makes it.

What this means for patients and clinicians#

The real value of a governance standard is that it converts a vague reassurance into a checklist a non-specialist can actually use. When a health system says it uses AI responsibly, that sentence now has a public reference point. You can ask which of the described elements are in place and expect a concrete answer. The 2025 guidance and the 2026 certification will not, by themselves, make every deployment safe. What they provide is a shared language for asking whether the safeguards are real, which is the first step before anyone can reasonably trust them.