How Regulators Use Real-World Evidence

Real-world evidence can support a regulatory decision when the underlying data are fit for that decision and the study design produces a credible comparison, and it can inform label changes, safety actions, postmarket commitments, device evaluations, and, in selected settings, effectiveness. It does not receive a pass merely because it contains many routine-care records.

The useful distinction is between real-world data and real-world evidence. Real-world data are observations collected during health care or everyday use. Examples are electronic health records, claims, and registries. Others are patient-generated data or device logs. Real-world evidence is the clinical evidence produced by analyzing those data for a defined question.

Key points#

• Regulatory acceptability is question-specific; one database may be fit for one decision and unsuitable for another.
• Relevance asks whether the needed patients, factors, outcomes, follow-up, and timing are captured.
• Reliability asks how data were collected, transformed, linked, curated, and quality-controlled.
• A prespecified causal design remains necessary when the question concerns effectiveness or comparative safety.
• Early discussion with the relevant agency can align the data and analysis plan with the intended submission.

Start with the decision, not the database#

A regulatory study should begin with a decision the evidence may change. Examples include whether to expand an indication, refine a label, or fulfill a postapproval requirement. Others are whether to compare long-term safety, support a control group, or confirm device performance after deployment.

That decision determines the evidentiary burden. A descriptive registry may document how a device is used. A comparative effectiveness claim requires a defensible counterfactual: what would have happened to similar patients under the alternative. Surveillance may prioritize sensitivity for a new safety signal, while a labeling decision needs enough specificity and bias control to estimate a risk.

Working backward from the decision produces a protocol: target population, treatment strategies, and time zero. The protocol continues with outcome, follow-up, and causal contrast. It ends with analysis and sensitivity tests. Working forward from a convenient database encourages questions chosen because the available columns happen to support them.

Relevance means the data contain the study you need#

FDA's device guidance frames relevance around whether the data address the regulatory question, which includes enough representative patients, clinically meaningful information, sufficient follow-up, and variables that can define the device, comparator, outcomes, and major confounders.

A claims database captures billed services across sites and can be strong for procedures and hospitalization, but weak for disease severity, laboratory results, symptoms, or why a clinician chose one treatment. Electronic records contain richer clinical detail but may lose care delivered outside the network, registries can standardize important fields but may enroll selected centers or patients, and device data can show actual use but not always the clinical context or outcomes.

Sample size is only one part of relevance. Ten million records do not compensate for an outcome that is never measured or a confounder that is systematically absent. Data must also come from a time and care environment compatible with the use you are proposing. Coding, guidelines, and product versions change.

Reliability is a chain of custody#

Reliability asks whether the data are sufficiently accurate and complete and whether their origin and transformations can be reconstructed. The chain begins where care or device use creates a record. It continues through coding, extraction, and linkage. It continues through cleaning, derivation, and analysis.

For each critical variable, you need an operational definition and validation evidence appropriate to its use, because a diagnosis code may have high positive predictive value in one health system and perform differently elsewhere. Death may be captured through hospital records, a national index, or family notification, each with different lag and completeness. A model-derived outcome needs version control and performance evaluation.

Data accrual should follow stable procedures. Missingness, implausible values, duplicate people, linkage error, and changes in software or reimbursement need quantitative checks; a polished common data model can improve consistency without fixing errors inherited from the source.

EMA's data quality framework emphasizes that quality is contextual. The 2026 chapter applying the framework to real-world data reinforces transparent assessment across data-generation and processing stages. A dataset is not globally “high quality.” It is more or less suitable for a specified use.

Limits imposed by design#

Routine-care treatment is not assigned at random. Clinicians choose therapies based on prognosis, contraindications, access, preference, and prior response. This confounding by indication can make an effective treatment appear harmful because it is given to sicker people, or make a favored treatment appear helpful because healthier patients receive it.

A credible non-interventional study specifies eligibility, treatment assignment, and time zero. It specifies follow-up, outcome, and analysis before looking at comparative results. Active-comparator, new-user designs often improve comparability by studying people at a similar decision point. Propensity scores, weighting, matching, outcome models, and doubly robust estimators can adjust measured differences. None corrects an important variable that was not captured or was measured badly.

Time-related bias deserves special attention. Immortal time occurs when a person must survive an interval to be classified as treated. Misaligned eligibility and treatment assignment can create it. Treatment changes, adherence, switching, and censoring require strategies matched to the intended effect.

Negative controls can reveal some residual bias. Quantitative bias analyses can show how strong an unmeasured confounder would need to be. Multiple databases can test transportability. These tools make assumptions more visible; they do not guarantee a causal answer.

Randomization and real-world data can coexist#

Real-world evidence is not synonymous with observational evidence. A pragmatic randomized trial can assign treatment while collecting outcomes from electronic records or registries. Randomization protects the comparison, and routine systems can reduce duplicate data collection and improve representativeness.

The tradeoff is that routine outcomes may be incomplete or less standardized, and implementation across ordinary sites can vary. The protocol still needs outcome definitions, data-quality checks, adherence assessment, and a plan for changes in care. The source of data does not determine the rigor of treatment assignment.

Externally controlled studies use real-world patients as a comparator for a single-arm trial. They may be important in rare diseases or settings where randomization is difficult. Their credibility depends on a common time origin, comparable eligibility, aligned outcome measurement, access to important prognostic variables, and evidence that changes in supportive care or diagnostic practice do not explain the contrast.

Where RWE already contributes#

FDA publishes examples across drug, biologic, and device decisions. Uses include adding or modifying indications, expanding populations, supporting postmarket surveillance, evaluating long-term outcomes, and deciding that no regulatory action is warranted. CDRH's 2025 final guidance updates how device sponsors should document relevance, reliability, study methods, and investigational-device considerations.

These examples are precedents, not templates that automatically validate the next submission. The same data source may support a device's postmarket safety analysis but not a new effectiveness claim, and a study can be scientifically informative while still falling short of the evidence needed for a specific legal standard.

Build an auditable submission#

An auditable RWE package should let a reviewer reconstruct:

1. the decision and estimand;
2. the data origin, governance, and permitted uses;
3. eligibility, treatment, comparator, time zero, and follow-up;
4. every critical variable and its validation;
5. linkage, cleaning, missingness, and transformation rules;
6. prespecified primary and sensitivity analyses;
7. protocol deviations and analyses added after results were known;
8. code, version history, and quality-control results.

Early agency interaction is valuable when your study will carry substantial regulatory weight. For devices, FDA specifically encourages sponsors to discuss planned RWD use through the Q-Submission pathway. For drugs and biologics, program-specific meeting mechanisms can clarify the question, while written agency advice does not guarantee a later approval.