Good Machine Learning Practice: 10 Principles for Trustworthy Medical AI

The short answer#

Good Machine Learning Practice (GMLP) is a set of ten guiding principles, published jointly in October 2021 by the U.S. Food and Drug Administration, Health Canada, and the United Kingdom's Medicines and Healthcare products Regulatory Agency, describing how a machine learning medical device should be built, tested, and monitored over its lifetime. They are not a law, and they are not a certificate a product can earn and then frame on the wall. They are closer to a shared checklist of good habits, agreed across three regulators, that answers one question a patient actually cares about: can this algorithm be trusted with my care, and is anyone still watching it?

Key points#

• GMLP is a voluntary, high-level framework of ten principles, not a regulation or an approval stamp.
• It follows the full lifecycle of a tool, from the first training dataset to the patient seen years after launch.
• The recurring themes are honest data, honest testing, honest labeling, and ongoing oversight.
• For a clinician or patient, the principles translate into three fair questions: Was it tested on people like me? Was it checked on data it had never seen? Is someone still monitoring it now?

Why a family doctor should care about a device framework#

Most of us will never write a line of model code, yet more and more of the tools around a clinic depend on one. A retinal camera that flags diabetic changes, a chest radiograph triage queue, a sepsis alert in the chart: each is a piece of software making a prediction, and each can be right in the demonstration and wrong at the bedside. GMLP matters because it gives non-engineers a vocabulary for asking whether that gap has been closed. The framework treats a medical algorithm the way careful clinicians treat any new test, asking where the evidence came from, whom it applies to, how it was checked, and who is accountable when it changes.

The principles overlap on purpose, and the order in which the agencies list them tells a story. Rather than repeat that list number by number, it helps to sort the ten into the four moments where trust is either earned or lost.

Moment one: who builds it#

Two of the principles govern the people and the plumbing behind a tool.

Multidisciplinary expertise across the lifecycle. A trustworthy medical AI team stretches well past its data scientists. It needs clinicians who know the problem, people who understand the workflow, and a clear sense of the patients the tool is meant to serve. That knowledge has to shape the project from the first meeting, not arrive at the end to bless a finished model.

Sound software engineering and security. Beneath the machine learning sits ordinary software, and ordinary software breaks in ordinary ways. This principle asks for disciplined engineering, data quality management, risk management, and cybersecurity, held to the same standard as any other device that can affect a patient. The clever part of a system is rarely where it fails first.

Moment two: what it learns from#

Three principles concern the raw material, because a model can only ever be as honest as the data behind it.

Representative data. A model mirrors its training population. If the people used to build it do not resemble the people who will later be scanned or screened, the tool can shine in the lab and stumble in the clinic. The principle asks that datasets reflect the intended patients, including relevant demographic and clinical variation, so that both the strengths and the blind spots are understood.

Genuine separation of training and test data. To know whether a model has learned something real, you have to test it on data it has never seen, kept truly apart from what shaped it. When the same patients, sites, or hidden signals bleed across both sets, the accuracy figures flatter the tool and mislead everyone downstream. Clean separation is what makes a validation result mean anything.

A trustworthy reference standard. Every performance metric is measured against something we agree to call the truth, the so-called ground truth. If that reference is inconsistent or biased, the error travels into every number that follows. The principle asks that the reference be well characterized and clinically defensible, because you cannot grade an answer against a blurry key.

Moment three: how it meets the real world#

Three more principles are about fit, the match between a tool and the setting where it will actually run.

A design matched to the data and the intended use. The model should suit the clinical problem, the data available, and the way the result will be acted on. A tool built for one population or one scanner can go wrong in another, so being explicit about the boundaries of its purpose is part of the design work, not a footnote.

The clinician and the AI as one team. Because a person almost always sits between the algorithm and the patient, what matters is how the pair performs together. This principle asks developers to study that partnership directly. Does the tool help the clinician catch more, or does it dull attention and invite automation bias, where a human defers to a confident machine? The unit being measured is the team, not the model alone.

Testing under clinically realistic conditions. A tool should prove itself in conditions that resemble real use rather than a tidy benchmark: the intended patients, the real equipment and sites, and the everyday workflow, evaluated with statistically sound study design on data that is independent of the training set. That is how a performance claim is earned rather than asserted.

Moment four: what users are told, and what happens next#

The final two principles reach past launch day.

Clear, essential information for users. Anyone relying on an output should understand what the model does, what it was trained and tested on, how it performs, and where it should not be used. The same agencies later described a companion set of transparency principles that expand on this idea. Plain labeling is a safety feature, because a tool nudged outside its intended conditions can become unreliable without anyone noticing.

Monitored deployment. A model is not finished when it ships. Real-world performance can drift as populations, practices, and equipment change, so a deployed system needs ongoing monitoring and maintenance. Related regulatory work, such as the idea of a predetermined change control plan, sets out how a manufacturer might describe planned updates ahead of time, instead of freezing a model forever or revising it invisibly.

What makes the framework hold up#

The strength of GMLP is that it treats an AI system as a clinical intervention with a lifecycle, not a product that is finished once and sold. Read in order, the principles form a chain: assemble the right people, engineer carefully, learn from data that looks like the real world, keep testing honest and separate, design for the actual task, measure human and machine as a team, evaluate under realistic conditions, tell users the truth, and keep watching. Break any single link and the whole chain can give way.

None of this requires believing that AI is either magic or menace. The principles sit at a deliberately high level, which is the point: they set direction while standards bodies and regulators work out the specifics, and international harmonization efforts have kept building on them. For a clinician or a patient, the practical residue is a short list of fair questions to put to any medical AI tool. Was it tested on people like me? Has anyone checked it on data it never saw during training? Is someone still watching it today?