A model that can read a note, interpret an image, hear speech, and generate a polished answer can feel less like software and more like a general colleague, and that impression is precisely why health organizations need a disciplined governance framework. Fluency is not evidence of clinical validity, and a single model can carry different risks when used for a discharge summary, symptom advice, radiology support, research synthesis, or medical education.
The World Health Organization's 2024 guidance on large multimodal models addresses this problem directly. It defines a large multimodal model, or LMM, as a generative AI model that can accept one or more data types, including text, images, and video, and generate diverse outputs. The document offers more than 40 recommendations for governments, developers, providers, and other actors.
The guidance is neither a product endorsement nor a universal technical standard. It is a governance map. Its practical value lies in forcing a health system to connect a model's claimed capability to a bounded use. That means accountable decision makers, evidence, and rights. It also means operational controls and continuing oversight.
Why multimodality changes the risk picture#
Earlier generative systems were often discussed as text tools. LMMs can connect several kinds of data. A system might combine a patient message with a photograph, summarize a recorded consultation, or generate a report from clinical text and an image. The ability to move across formats expands possible use, but it also creates more ways for an error to enter or become difficult to trace.
An image may be cropped, compressed, mislabeled, or acquired with an unsupported device. Speech recognition may distort an accent, medication name, or number. A note may include copied material that no longer applies. The model can then combine these inputs into one confident answer, making source-specific uncertainty disappear.
Multimodality also complicates privacy. A photograph may contain a face or location. Audio may capture a family member. Document metadata may reveal identity. A procurement team that evaluates only the typed prompt can miss data flows created by other input modes, logging, model improvement, or vendor support.
The correct unit of assessment is therefore not “the model.” It is the complete application in a defined environment. That includes interfaces, data pathways, and users. It includes handoffs, overrides, downstream actions, and maintenance.
The five broad applications WHO describes#
The first application is diagnosis and clinical care, such as responding to written patient questions or supporting a clinical assessment. The harm threshold is high because output may influence testing, treatment, triage, or delay.
The second is patient-guided use. People may ask about symptoms, medicines, or treatment without a clinician present. The model may not know the person's full history, may fail to detect an emergency, and may produce language that sounds more personal than the evidence permits. Accessibility can improve, but access to a persuasive tool is not the same as access to safe care.
The third is clerical and administrative work, including drafting notes or summarizing visits; these uses may appear lower risk, yet a wrong allergy, omitted negation, invented examination finding, or incorrect recipient can propagate through the record. Administrative efficiency must be assessed alongside documentation accuracy and correction burden.
The fourth is medical and nursing education. Simulated patients, tutoring, and feedback can broaden learning. The model can also teach an outdated rule, flatten uncertainty, or generate a false citation. Educators need a reviewed curriculum and a way to distinguish practice material from authoritative instruction.
The fifth is scientific research and drug development, including generating hypotheses or identifying candidate compounds. Risks include fabricated references, hidden data leakage, irreproducible analysis, biased prioritization, and overstatement of exploratory results. Research governance and validation do not disappear when generation becomes faster.
Fluency is not reliability#
WHO highlights false, inaccurate, biased, and incomplete outputs. These are especially dangerous when the model presents them in a coherent narrative. You may read polished language as calibrated confidence even when the model has no dependable estimate of its own uncertainty.
Traditional software often fails visibly. A generative model can fail gracefully in style while failing materially in content: it may give the right recommendation for the wrong reason, invent a supporting fact, omit a contraindication, or change its answer after a minor prompt variation.
Evaluation must therefore go beyond overall accuracy. It should examine severe-error rates, unsupported claims, and omissions. It should examine calibration, subgroup performance, and prompt sensitivity. It should examine refusal behavior, citation validity, and the consequences of incorrect output. The acceptable threshold will differ for a brainstorming aid and a triage message.
A benchmark assembled from clean textbook examples may not represent short patient messages, mixed languages, or poor images. It may not represent incomplete records, rare disease, disability-related communication, or local practice. External and local testing matter because a model's average performance can conceal weak performance in the very circumstances where help is most needed.
Data quality, bias, and inequity#
An LMM learns patterns from large data collections whose provenance may be partly opaque. Historical records can contain unequal access, diagnostic delay, stigmatizing language, underrepresentation, and labels that encode prior clinical decisions rather than biological truth.
Bias can arise before model training, during model design, at interface level, or in deployment. A symptom checker may be less usable with a screen reader. Speech performance may vary across accents. A skin-image system may have fewer examples across skin tones. A summarizer may repeat a prejudicial statement from an old note and give it new authority.
Equity assessment needs more than one demographic table. Ask who can access the tool, whose data were used, whose language is supported, who receives a fallback service, how an error reaches you, and whether the tool shifts work onto patients or staff with fewer resources.
Disaggregated monitoring can reveal disparities, but sensitive attributes also require lawful, privacy-protective handling. Governance should define why data are collected, how they are secured, who can analyze them, and how findings trigger remediation.
Why “a human checks it” is incomplete#
Human review is often named as the main safeguard. It can help, but only within a designed workflow. A clinician faced with hundreds of plausible drafts may develop automation bias and stop checking each detail. A patient may not know what needs verification. A junior worker may detect an error but lack authority to reject a model's output.
Effective oversight specifies the reviewer, information available, and time allowed. It specifies decision rights, escalation route, documentation, and accountability. It measures whether reviewers actually catch seeded errors under realistic workload. It also protects the ability to complete the task without the model when the system is unavailable or unsafe.
The phrase “human in the loop” can conceal a transfer of liability without a transfer of control. If the interface hides source text, makes edits cumbersome, or rewards speed, formal review may become a ritual. Usability testing and workload measurement are safety work.
Recommendations for governments#
WHO asks governments to create conditions in which public benefit and rights shape development. Recommendations include investing in public or nonprofit infrastructure, using law and policy to protect dignity, autonomy, and privacy, assigning regulatory capacity, and requiring post-release auditing and impact assessment for large-scale deployment.
Public infrastructure can reduce dependence on a small number of private vendors and make high-quality datasets or computing resources available under ethical conditions. Yet public ownership is not itself a safety guarantee. Governance, security, representation, and accountability still apply.
Regulatory responsibility should match intended use and risk. A system that performs a medical-device function may fall under medical-device law. A general model embedded in a clinical product may acquire new obligations through the product's claims and workflow. Privacy, consumer protection, and civil rights can apply at the same time. So can professional regulation, procurement law, and cybersecurity.
WHO's document is not itself binding national law. A country may adopt, adapt, or differ from its recommendations. Map the guidance to your actual jurisdiction rather than treating a citation to WHO as compliance.
Recommendations for developers and providers#
WHO calls for structured participation by patients, health workers, researchers, civil society, and other affected groups from early design onward. Participation should influence the use case, interface, success criteria, and stop conditions, not merely validate a finished product.
Developers should define tasks narrowly and demonstrate necessary accuracy and reliability, and they should document training and evaluation methods as far as rights and legitimate confidentiality permit, examine foreseeable secondary consequences, build cybersecurity controls, and consider environmental cost.
Providers and health organizations need their own assessment even when a vendor supplies extensive evidence. You choose the population, workflow, permissions, fallback, and response to error. A model can be used outside its tested purpose through a small interface change or an informal staff habit.
Contracts should address data use, model changes, and incident notification. They should address audit access, performance evidence, and subcontractors. They should address retention, termination, and export of records. A vendor's right to alter a model can undermine your validation unless change control is explicit.
A practical pre-deployment evidence file#
Begin with an intended-use statement: who uses the tool, for whom, for what decision, with which inputs, and what action may follow. List excluded uses. A broad phrase such as “clinical assistant” is not testable.
Create an evidence table. Include model and application version, development data provenance, evaluation sets, reference standards, performance by relevant subgroup, severe failure examples, uncertainty behavior, security testing, privacy impact assessment, accessibility testing, and local workflow simulation.
Map every output to an accountable person and a response. Who corrects the record? Who contacts a patient after a harmful message? Who can suspend the system? What happens when the model, network, or vendor is unavailable? How are incidents preserved for review?
Set launch gates. A pilot should have limited users, bounded cases, prospective monitoring, and predefined stopping rules. Efficiency gains should not be counted without measuring correction time, displaced work, new inequities, and downstream utilization.
Monitoring after release#
Model performance can change even when the underlying model does not. Patient populations, clinical policy, and disease prevalence evolve. So do connected systems, prompts, and staff behavior. Vendor updates can change outputs directly.
Post-deployment monitoring should track meaningful outcomes, not only uptime and user satisfaction. Useful measures include material correction rate, serious omissions, and unsupported claims. They include override rate, subgroup differences, and patient complaints. They include security events, delayed care, and fallback use.
Every metric needs an owner, review frequency, threshold, and response. A dashboard that no one has the authority to act on is not governance. Incident review should lead to corrective action, communication, revalidation, restriction, or withdrawal as appropriate.
Decide in advance what evidence would persuade you to expand. Success in drafting low-risk messages does not establish readiness for triage. Each new use creates a new claim that needs assessment.
How other frameworks fit#
The NIST AI Risk Management Framework organizes work around governing, mapping, measuring, and managing risk. The IMDRF good machine learning practice principles address total-product-lifecycle practices for AI-enabled medical devices. WHO's regulatory considerations describe issues regulators may consider, while explicitly not serving as a complete regulatory framework.
These resources overlap but are not substitutes. Use WHO for ethical and institutional questions, NIST for a general risk-management vocabulary, IMDRF for device development practice, and applicable law for enforceable duties. The right combination depends on what you are actually deploying.
WHO's 2026 discussion paper on AI in evidence-informed health policy reinforces the same direction: generative systems can accelerate synthesis and communication, but transparency, inclusion, and trustworthy evidence processes remain necessary.
References#
- WHO guidance on ethics and governance of large multimodal models
- WHO summary of large multimodal model recommendations
- WHO ethics and governance of artificial intelligence for health
- WHO regulatory considerations on artificial intelligence for health
- NIST Artificial Intelligence Risk Management Framework 1.0
- IMDRF good machine learning practice guiding principles
- WHO discussion paper on AI in evidence-informed health policy
Questions and answers
What is a large multimodal model?
It is a generative AI model able to accept one or more input forms, such as text, images, audio, or video, and generate varied outputs. The label describes a technical capability, not clinical validation or regulatory status.
What five health uses does WHO describe?
WHO describes diagnosis and clinical care, patient-guided use, clerical and administrative work, medical and nursing education, and scientific research and drug development. Each category contains many distinct risk levels.
Is WHO's guidance a law or an approval for a product?
No. It is global normative guidance. National and local law, medical-device rules, privacy requirements, professional duties, procurement terms, and the exact product's regulatory status still apply.
Does a clinician reviewing every output make an LMM safe?
Not by itself. Reviewers need enough time, information, training, authority, and a usable correction path. The organization should test whether oversight works under real workload and should retain a safe fallback.
What evidence should a health organization request before deployment?
It should request evidence tied to the exact use and population, including subgroup performance, material failure modes, data and security controls, local workflow results, accessibility, change control, incident response, and a post-release monitoring plan.