The Cost of a False Positive

A false positive occurs when a test, model, monitor, or rule calls a target condition present even though the chosen reference says it is absent. That definition appears simple. The consequences are not.

The result may be repeated, confirmed, copied into a record, or converted into a referral. You may receive imaging, biopsy, medicine, isolation, or monitoring. A clinician may spend time resolving the signal and less time on another problem. In software, one false alert can be dismissed, while thousands can reshape an entire service.

The cost is therefore a pathway, not a number printed beside specificity. To judge it you have to know what the test is for, what counts as positive, what the reference standard is, where the threshold sits, and how common the condition is in the people being tested. Then you have to know what happens next: which action a positive call triggers, whether it can be undone, how long it takes, and who ends up bearing the error.

Begin with the four cells, then leave the table#

For a binary test, results can be organized as true positive, false positive, true negative, and false negative. Sensitivity is the proportion of reference-positive cases that the test calls positive. Specificity is the proportion of reference-negative cases it calls negative.

The false-positive rate is one minus specificity. If specificity is 95%, the false-positive rate is 5% among cases considered negative by the reference standard, though that is not the same as saying that 5% of the positive calls you see are wrong.

The proportion of positive calls that are correct is positive predictive value, or PPV:

PPV = true positives / (true positives + false positives)

PPV changes with the prevalence of the target condition in the population being tested. Sensitivity and specificity may also change across settings because of spectrum, measurement, reader, and threshold differences, but prevalence has a direct mathematical role in predictive value.

The confusion matrix is the beginning because it counts classifications. It is not the end because two false positives can have radically different consequences. A repeat swab and an invasive procedure do not carry the same burden.

A low-prevalence example#

Imagine a condition present in 1% of 10,000 people. Suppose a test has 90% sensitivity and 95% specificity.

Among 100 people with the condition, about 90 would receive a true-positive result and 10 a false-negative result. Among 9,900 people without it, 5% would test positive, producing about 495 false positives.

The test would generate 585 positive calls, of which 90 are true positives. The PPV is about 15%. Most positive calls would be false, even though 95% specificity sounds strong when you read it in an abstract.

This example does not prove the test is useless. If the condition is catastrophic, confirmatory testing is safe, and early treatment is highly effective, accepting many false positives may be reasonable. If the next action is risky, irreversible, or scarce, the same numbers may be unacceptable.

Prevalence also changes along a care pathway, and a test used after clinical triage in a specialty clinic may have a higher PPV than the same test used as a broad population screen. A performance claim should make clear the population and where in the workflow the test sits.

A false positive needs a reference standard#

Calling an output false assumes we know the truth. Often we use a reference standard that is imperfect, delayed, subjective, or only a proxy for the target condition.

A radiology model may flag a lesion that the original report omitted. Later pathology might confirm the model, making the supposed false positive a reference error. A sepsis alert may be judged against billing codes that miss clinical cases. A pathology reference may use consensus among readers who disagree at the boundary.

The target condition must also be defined. Does “positive” mean any imaging abnormality, disease requiring treatment, an outcome within 48 hours, or a clinician's decision to intervene? A model can be correct for one target and unhelpful for another. So a good diagnostic study will tell you how the reference was established, whether readers knew the index result, how indeterminate cases were handled, and how long outcomes were followed; the 2025 STARD-AI guideline extends those reporting expectations to diagnostic-accuracy studies involving AI.

Direct medical and procedural costs#

A positive result often starts verification. The next action might be another laboratory test, repeat imaging, specialist review, endoscopy, biopsy, monitoring, or empiric treatment. Each can add direct financial cost, travel, time away from work, and procedural risk.

Confirmatory testing may itself be imperfect. A chain of tests creates more opportunities for incidental findings and discordance. An incidental finding can start a second chain unrelated to the original question.

Unnecessary treatment can cause adverse effects and interactions. Even a low-risk intervention creates documentation, follow-up, and a chance that later clinicians interpret the treatment history as proof that the diagnosis was established.

The relevant estimate is not simply “cost per test.” It is the expected cost of the downstream branch multiplied by how often the branch is entered, plus the benefit from true-positive branches and the harm from missed cases.

Psychological, social, and informational costs#

An abnormal result can change how a person sees their own health before anyone has resolved it. Anxiety may be brief, or it may persist after a negative follow-up. Waiting can be a substantial burden when the word said out loud is cancer, genetic disease, pregnancy complication, or infection.

Labels can affect insurance processes, employment decisions where law permits, family dynamics, reproductive choices, and eligibility for procedures; a result copied into a problem list may outlive the evidence that refuted it.

Privacy harm can arise when a false label is shared across systems or portals. Correction is often harder than addition. Structured fields, coding, and automated summaries can repeatedly surface an outdated diagnosis. Researchers should measure these outcomes directly when they matter, because counting follow-up tests alone misses the distress, the time cost, and the effort it takes to get a record corrected.

Opportunity cost and diagnostic distraction#

Every alert consumes some attention. Reviewing a chart, contacting a patient, ordering a test, and documenting a decision use finite clinical time. At scale, even a short review can become a large staffing burden.

False-positive anchoring can also redirect the diagnostic search. A clinician may organize symptoms around the erroneous result and give less weight to the actual cause. The AHRQ diagnostic pathway brief emphasizes pretest probability, post-test probability, and decision thresholds because a test result should update reasoning rather than replace it.

Opportunity cost is not just monetary. A scanner used for unnecessary follow-up is unavailable for someone else. A referral queue grows. A nurse responding to low-value alerts has less time for medication reconciliation or symptom calls. A patient may delay another appointment while pursuing the false lead.

Alert fatigue is a system response#

When users repeatedly see alerts that do not help, they adapt. They may dismiss faster, create workarounds, or stop distinguishing meaningful signals. The system's effective sensitivity can then fall even if the model's technical sensitivity is unchanged.

Alert fatigue is not a moral defect in the user. It is evidence that signal design, threshold, timing, routing, or actionability may be poorly matched to workflow. Measuring only whether an alert fired omits whether it was seen, understood, accepted, acted upon, and beneficial.

An alert can be statistically correct yet operationally false for the intended decision. For example, predicting an event that has already become clinically obvious may add no useful lead time. An output delivered to someone unable to act has little value. Human factors evaluation therefore has to examine interruption, display, uncertainty, explanation, escalation, staffing, and failure recovery; FDA's transparency principles emphasize intended users, use environment, workflow, limitations, and the performance of the human-AI team.

Overdiagnosis is related but different#

A false positive says the target is absent according to the best available truth, and overdiagnosis means a condition is truly detected but would never have caused symptoms or harm during the person's life. The result is technically true and clinically unnecessary.

Both can lead to extra testing and treatment. They require different evidence. Better test specificity can reduce false positives but may not reduce overdiagnosis if the test becomes more sensitive to indolent disease.

Incidental findings form another category. They may be real and important, real and harmless, uncertain, or false. A screening program should anticipate how they will be communicated and resolved rather than treating them as free information.

Thresholds convert scores into actions#

Many tests and models output a continuous value or probability. A threshold turns that value into a positive call, an alert, or a treatment recommendation. Lowering a threshold usually increases sensitivity and false positives. Raising it usually increases specificity and false negatives.

There is no universally optimal threshold, because the right tradeoff depends on the consequences: missing a reversible, rapidly fatal condition may justify a low threshold, while sending someone for an invasive procedure may require a higher probability first.

Thresholds should be tied to an action. A score of 0.12 is not inherently positive; it might justify watchful waiting, a noninvasive confirmation, a specialist referral, or no change, depending on the setting and patient preference. Optimizing Youden's index or choosing the upper-left point of a receiver-operating curve weights errors in a mathematical way that may not match anyone's clinical values, so the decision should state whose harms and benefits count, and over what time horizon.

Why area under the curve is not enough#

Area under the receiver-operating characteristic curve, often called AUROC, measures how often a randomly selected positive case receives a higher score than a randomly selected negative case; it summarizes ranking across all possible thresholds.

AUROC does not tell you about calibration, prevalence, workload, or the consequences at the threshold you will actually use. A small AUROC improvement can be useless at the operating range. A model with a lower AUROC can be better calibrated and more useful for a specific decision.

Precision-recall curves can be more informative for rare targets because precision corresponds to PPV, but they still do not assign value to the resulting actions. Calibration asks whether predicted risks correspond to observed frequencies, and it is essential when the numeric probability guides a threshold. So what you want reported is the confusion matrix at relevant thresholds, the predictive values with the study prevalence, the uncertainty intervals, the calibration, and the counts per meaningful unit. Then show what happened after the output.

Net benefit makes the tradeoff explicit#

Decision-curve analysis evaluates a strategy across threshold probabilities; it counts true positives and subtracts false positives weighted by how a decision maker values the harm of an unnecessary action relative to the benefit of a correct one.

The method compares a model with default strategies such as acting for everyone or no one, and a model can have positive net benefit over one threshold range and no value outside it. The net-benefit framework translates statistical performance toward a clinical decision without pretending that all consequences are identical.

Net benefit is not a universal moral calculator. The chosen threshold encodes preferences and consequences. Different patients or systems may reasonably choose different thresholds. Severe adverse effects, unequal access, multiple downstream actions, and budget constraints may require fuller decision or cost-effectiveness analysis.

Unit of analysis can hide burden#

Suppose an imaging model reports 95% specificity per image. If each examination contains hundreds of images, the chance of at least one false flag per examination may be much higher. A lesion-level false-positive rate cannot be read as a patient-level rate.

Similarly, one person may generate many vital-sign observations and alerts during a hospital stay, and reporting accuracy per observation can make a system seem stable while clinicians face repeated false alerts per patient-day.

Clustering also affects uncertainty. Images within a person, admissions within a hospital, and repeated visits within a person are not independent. Confidence intervals and test sets should respect that structure, and splitting records from the same person across training and test data can produce data leakage and inflated results. Whatever the structure, the denominator should match the decision: per patient screened, per completed examination, per alert delivered, per clinician shift, or per 1,000 patient-days.

Subgroup errors can carry unequal costs#

An overall false-positive rate can conceal higher rates in groups defined by age, sex, skin tone, disability, language, device type, site, disease severity, or care setting. Smaller subgroups also yield wider uncertainty.

Unequal false positives can concentrate invasive follow-up, stigma, or denial of service in one population. Equal rates do not necessarily mean equal harm if downstream access and treatment differ.

Subgroup analysis should be prespecified where possible, based on plausible failure mechanisms, and reported with sample sizes and uncertainty. Endless exploratory slices can manufacture apparent differences. A fairness claim requires both statistical and clinical interpretation.

Site-specific factors matter too. Scanner vendor, laboratory platform, documentation pattern, prevalence, and referral practice can shift performance. Validation at one hospital does not establish transportability everywhere.

Retrospective accuracy is not clinical impact#

A retrospective study can show that a model distinguishes archived cases. It cannot show how users will respond, whether care improves, or whether new harms appear after deployment.

The DECIDE-AI guideline addresses early live clinical evaluation of AI decision support, including human factors and system use. Larger comparative studies may then assess patient outcomes, workflow, and resource use.

Silent evaluation, in which outputs are generated but not shown to clinicians, can estimate local alert volume and technical performance before active use. It still cannot measure behavior after display. A stepped or randomized rollout may provide stronger evidence when feasible. And monitoring cannot stop at deployment, because patient mix, workflow, coding, sensors, and treatments all change; the 2025 IMDRF Good Machine Learning Practice principles frame performance across the total product life cycle.

A complete false-positive ledger#

For each positive output, a useful evaluation can record:

1. The target and reference result.
2. The threshold and model version.
3. Who received the output and when.
4. Whether it was acknowledged, overridden, or accepted.
5. Tests, referrals, procedures, or treatments that followed.
6. Time to resolution and any delay in other care.
7. Adverse events, distress, and out-of-pocket burden.
8. Record labels created or removed.
9. Staff time and capacity used.
10. Whether the same person received repeated alerts.

The ledger also needs true positives, true negatives, and false negatives. Reducing false positives by silencing every alert would be a bad success. The goal is better decisions and outcomes, not one optimized cell.

Design choices that can reduce harm#

Start with a precise intended use and a consequential target. Validate data provenance and avoid leakage. Evaluate calibration and counts at clinically plausible thresholds. Use confirmation when it reduces harm more than it adds burden.

Route outputs to someone able to act. Display uncertainty and the reason for the alert when useful. Avoid interruptive presentation when a batched or passive view is safer. Suppress duplicates when repeated signals add no information.

Provide a clear resolution pathway. If users cannot determine how to close an alert, the false-positive burden persists. Monitor overrides and downstream actions, but do not equate acceptance with correctness.

Build a stop rule and rollback plan. A system should be paused when safety, workload, drift, or inequity crosses a defined boundary. Transparency about known failure modes is a safety feature, not an admission of defeat.

The decision-centered conclusion#

The cost of a false positive begins with an incorrect call and unfolds through a health system. It can be a repeat test, an invasive procedure, a durable label, a frightened week, an occupied appointment, or attention diverted from a true diagnosis.

Good evaluation follows that chain. It uses the right denominator, tests the intended setting, recognizes imperfect references, and weighs errors at a real decision threshold. For AI, it evaluates the human-system team before and after deployment.

A useful test does not merely classify well. It improves decisions enough that the benefits of correct action exceed the harms, burdens, and missed alternatives created by using it.

References#

• Agency for Healthcare Research and Quality. Probability and the diagnostic pathway.
• National Academies of Sciences, Engineering, and Medicine. Improving Diagnosis in Health Care. 2015.
• Vasey B, et al. DECIDE-AI reporting guideline. Nature Medicine. 2022.
• Sounderajah V, et al. STARD-AI reporting guideline. Nature Medicine. 2025.
• Vickers AJ, et al. Net benefit approaches to evaluation of prediction models and tests. 2016.
• FDA. Transparency for machine-learning-enabled medical devices.