Missing Trial Data Are an Assumption Problem, Not an Empty-Cell Problem

Missing data threaten a trial when participants with unobserved outcomes differ systematically from those who remain observed. Filling blank cells is not the main challenge. The challenge is deciding which untestable assumptions connect the observed records to outcomes that were never measured.

Key points#

• The amount, timing, reasons, and treatment-group pattern of missing data matter more than one overall percentage.
• Complete-case analysis is unbiased only under restrictive conditions and usually sacrifices precision.
• Multiple imputation represents uncertainty across several plausible completed data sets, but its validity depends on the imputation model and missingness assumptions.
• Last observation carried forward treats a guessed value as known and can bias results in either direction.
• Sensitivity analyses should examine departures from the main assumption, especially when missingness could depend on unobserved outcomes.

Begin with the question the trial intends to answer#

Before choosing an imputation method, a trial needs a precise target of estimation. The ICH E9(R1) framework calls this target an estimand. It describes the treatment conditions being compared, population, outcome variable, approach to events after randomization that affect interpretation or measurement, and population-level summary such as a mean difference or risk ratio.

Those post-randomization events, called intercurrent events, include treatment discontinuation, rescue medication, treatment switching, and death before outcome assessment. They are not automatically missing data. A participant may stop assigned treatment but still provide the planned outcome, allowing an analysis of the effect of assignment regardless of discontinuation. Conversely, a value may be absent even though no intercurrent event occurred.

The distinction prevents a common mistake: choosing a convenient data method before deciding which clinical question it is supposed to answer. An analysis about outcomes regardless of discontinuation needs continued follow-up after treatment stops. A hypothetical question about outcomes had discontinuation not occurred requires a different model and stronger assumptions.

A percentage missing does not describe the risk#

Two trials can each report 12% missing outcomes and face very different threats.

In one, missed assessments may be balanced across groups, occur for administrative reasons, and have strong predictors in earlier measurements. In another, participants may leave one group because symptoms worsened or adverse effects occurred. The second pattern is more likely to bias the treatment comparison.

A useful report includes:

• missingness by randomized group and visit,
• reasons for missing assessments,
• whether assigned treatment was discontinued,
• which outcomes were still collected after discontinuation,
• baseline and earlier-outcome differences between observed and unobserved participants,
• and whether deaths or other undefined outcomes were handled separately.

Reasons labeled “withdrew consent” or “lost to follow-up” may be too broad. They tell you little about whether the unobserved outcome was likely better or worse.

MCAR, MAR, and MNAR are model assumptions#

The standard vocabulary describes how missingness relates to data.

Missing completely at random#

Under MCAR, the probability that a value is missing is unrelated to observed or unobserved information relevant to the outcome. An unpredictable equipment failure might approximate this condition. MCAR is strong and uncommon in longitudinal health studies.

Missing at random#

Under MAR, missingness may depend on observed information but not on the missing value after conditioning on that information; for example, absence at the final visit might depend on earlier symptom scores, provided those earlier scores are included appropriately in the model.

MAR does not mean the records vanished randomly in everyday language. It is a conditional assumption. Its plausibility improves when the analysis includes strong predictors of both outcome values and missingness.

Missing not at random#

Under MNAR, missingness still depends on the unobserved value after accounting for observed data, and a participant whose symptoms became much worse may be more likely to miss the final assessment even after earlier symptoms are considered. Observed data generally cannot prove whether MAR or MNAR is true, because the missing values are unavailable. Clinical knowledge, reasons for absence, and sensitivity analyses must carry part of the argument.

Why simple deletion and single guesses are fragile#

A complete-case analysis discards anyone missing a variable required by the model, which reduces information and can upset the protection of randomization when exclusion is related to prognosis or treatment. It can be valid in particular settings, but “we analyzed available cases” is not a sufficient justification.

Single imputation inserts one replacement value and then analyzes the data as though that value were observed; mean imputation understates variation and weakens relationships among variables, and best-case and worst-case replacement can be useful as extreme sensitivity checks but not as realistic primary analyses.

Last observation carried forward copies a participant's most recent value into every later missing visit. It assumes no later change and treats the copied number as certain. Symptoms may improve, worsen, fluctuate, or respond differently after treatment stops. The method can favor either group depending on the disease course and timing of dropout, so it is not inherently conservative.

What multiple imputation does#

Multiple imputation creates several completed data sets. Each missing value is replaced with a draw from a predictive distribution based on the observed information. The planned analysis is run in every data set, and the estimates are combined using rules that incorporate both variation within each analysis and variation between imputations.

That between-imputation variation is important. It acknowledges that the true missing value is uncertain. A single filled-in data set cannot represent this uncertainty adequately.

A credible imputation model generally includes randomized group, variables in the analysis model, earlier measurements of the outcome, and predictors of missingness or the missing value, and it should respect variable types and plausible ranges. Interactions or nonlinear relationships important to the analysis may also need representation. The number of imputations should be adequate for the fraction of missing information and desired numerical stability.

Multiple imputation is not a magic repair. Standard implementations often rely on MAR. A poorly specified model, weak predictors, incompatible transformations, or very high missingness can produce precise-looking but unreliable estimates.

Sensitivity analysis asks what the unseen values would need to be#

The primary analysis uses one set of assumptions. A sensitivity analysis varies assumptions that cannot be verified from observed data while preserving the same estimand.

Pattern-mixture approaches can shift imputed outcomes for participants with missing data relative to MAR predictions. A tipping-point analysis increases or decreases that shift until the conclusion changes. Reference-based methods may model outcomes after discontinuation using patterns from another randomized group, but the clinical meaning of that choice must be explained.

The most informative sensitivity analysis uses departures that clinicians and patients can understand. Instead of presenting only technical model names, a report can tell you how much worse the unobserved outcomes in one group would need to be to erase the estimated benefit. If a small plausible shift reverses the result, the conclusion is fragile. If only an extreme shift does, confidence is stronger.

Sensitivity analysis should not change the scientific question without acknowledgment. A model answering a hypothetical no-discontinuation question is not a sensitivity analysis for an assignment-regardless-of-discontinuation estimand. It is a different estimand.

Prevention has more value than statistical repair#

Good trial design separates follow-up from continued treatment. Participants who stop assigned treatment can often still contribute outcome data if consent and safety permit. Short, focused follow-up forms, flexible visit options, updated contact methods, clear retention procedures, and collection of reasons for missed visits can reduce missingness. The National Research Council emphasized specifying objectives, limiting unnecessary data collection, and continuing outcome collection for discontinued participants when relevant, and these measures provide information that no later model can recreate.

A reader's checklist#

When you read a trial with incomplete outcomes, ask:

1. What estimand was the analysis intended to estimate?
2. How much outcome data were missing in each group and at each time?
3. Why were values missing, and are the categories informative?
4. Were participants followed after treatment discontinuation?
5. Does the primary method rely on MCAR, MAR, or another assumption?
6. Does the imputation model include earlier outcomes and predictors of absence?
7. Were imputed values and model diagnostics checked for plausibility?
8. Were deaths and outcomes that cannot be defined handled explicitly?
9. Did sensitivity analyses examine clinically plausible MNAR departures?
10. Does the conclusion survive those analyses?

Missing data do not automatically invalidate a trial. Hidden assumptions do. A trustworthy report states those assumptions, aligns them with the treatment question, and shows how much the answer depends on values that were never observed.