"Peer reviewed" is often used as a dividing line between trustworthy and untrustworthy evidence. The distinction matters, but the label carries less information than it seems. Review practices vary, reviewers work from a limited record, and editors make the final decision. A published paper can be rigorous, mistaken, incomplete, or fraudulent. An unreviewed preprint can be informative, but its claims have not passed through the journal's review process.
Peer review is best understood as organized criticism before an editorial decision. It can improve clarity, methods, analysis, and interpretation. It cannot reproduce the study, inspect every underlying observation by default, or certify that the conclusion will hold in other data.
The journal pathway#
A manuscript first reaches an editor or editorial team. This initial screen can consider subject fit, importance, reporting completeness, ethics documentation, originality, and whether the work meets basic methodological expectations. Many submissions are declined without external review.
If sent for review, the editor selects people with relevant expertise. They assess the manuscript and recommend acceptance, revision, or rejection, often with detailed comments. Authors respond and may revise through several rounds. The editor weighs those reviews, journal policy, and editorial judgment before deciding.
ICMJE guidance is clear that reviewers advise editors; their recommendations are not votes that bind the journal, and reviewers may disagree, focus on different issues, or lack expertise in one part of a multidisciplinary study. An editor can request another review, statistical assessment, protocol review, or additional analysis.
Review models differ. Author or reviewer identities may be concealed, partly disclosed, or open. Some journals publish reviews and responses. The number of reviewers, use of statistical reviewers, availability of protocols, requirements for data and code, image screening, and conflict-of-interest procedures vary. The phrase "peer reviewed" does not tell you which safeguards were used.
What reviewers can examine#
Reviewers can ask whether the design matches the research question. A causal claim from a cross-sectional association, for example, warrants challenge. They can examine inclusion criteria, comparison groups, outcome definitions, missing-data methods, multiplicity, model assumptions, and whether uncertainty is reported.
They can also check internal coherence. Do participant counts reconcile across the abstract, flow diagram, tables, and results? Are methods described for the outcomes reported? Do effect estimates match confidence intervals? Does the conclusion reflect the prespecified primary outcome, or shift attention to a favorable secondary analysis?
Reporting quality is part of interpretability. Review can request clearer intervention descriptions, fuller harms reporting, protocol registration, funding and conflict disclosures, and a more balanced account of limitations. Reviewers can identify overlooked literature or a technical problem within their expertise.
The word "can" is important. Review does not reliably catch every issue in these categories. Reviewers have limited time, manuscripts can be complex, and expertise is never complete. A suggestion can also be wrong. Author revision may improve one aspect while introducing another.
What reviewers usually cannot see#
Reviewers ordinarily see a manuscript and supplementary material, not every action taken during the research. Unless protocols, analysis plans, data, and code are supplied and examined, they may not know that an outcome was omitted, a model was chosen after many alternatives, or a subgroup result was selected because it looked favorable.
Selective outcome reporting has been documented by comparing trial protocols with publications. Outcomes can be added, omitted, or changed in prominence. Registration and protocol access make these changes easier to detect, but they do not eliminate them. Compare dates and prespecified outcomes yourself when the decision stakes are high.
Underlying data can contain errors that do not reveal themselves in summary tables. Fabricated or altered data may look plausible. Image or statistical screening can flag some patterns, and data-sharing requirements can expand scrutiny, but peer review is not a forensic audit by default.
Reviewers can also miss a problem outside their specialty. A clinical reviewer may not recognize an advanced modeling flaw; a statistician may lack domain knowledge needed to see that an outcome definition is implausible. Multidisciplinary review can reduce this gap without removing it. And no reviewer repeats the study: nobody can know from the manuscript alone whether another team, dataset, laboratory, or setting will obtain a compatible result.
Evidence that review misses errors#
In a randomized study reported by Schroter and colleagues, 607 reviewers assessed a manuscript into which nine major errors had been inserted; reviewers detected only some errors, and no reviewer found all nine. Training interventions produced limited improvement in that setting.
A later Cochrane review evaluated interventions intended to improve peer review; it found low- or very-low-certainty evidence for many outcomes and did not establish a simple training method that makes review reliably detect all important problems. Differences among studies, outcomes, and journal processes limit broad conclusions.
These findings do not show that peer review is useless. They show that review is a fallible quality-control process. It can add scrutiny and lead to substantial revision while leaving residual error. The right comparison is not perfection versus failure, but what review adds and what further checks remain necessary.
What a preprint is#
A preprint is a manuscript made publicly available before completion of journal peer review. A server may screen for scope, basic policy compliance, harmful content, or obvious problems. That screen is not the same as journal review by subject and methods experts.
Preprints can make results available quickly, invite criticism, establish a time-stamped record, and support rapid scientific exchange. Speed also means that methods, analyses, and wording may change. In medicine, premature public interpretation can influence care or policy before claims are adequately checked.
ICMJE guidance advises authors to identify preprints, inform journals, and update the preprint record with the later publication when applicable. Look for a journal version rather than assuming the first search result you find is the current one. And call a preprint a preprint, in headlines, summaries, and citations: describing it as "a study published in" a repository implies a journal validation that never happened.
Versioning changes what is being evaluated#
Preprint servers can host several versions. The title, author list, analysis, sample, conclusions, and limitations may change. Record the version date and number so that you know which text you are citing, and compare it with any later journal article.
The journal version is not always identical to the preprint. Review may prompt reanalysis, removal of an unsupported claim, added data, or more cautious language. Sometimes the central result changes, and when a peer-reviewed version exists, it is usually the appropriate version to cite for the reviewed claims, while material differences from the preprint can remain relevant.
Published articles also have versions. Online-first text can be corrected before issue assignment, and later formal corrections can amend the record. A link to a static copy may miss those changes. Persistent identifiers and the journal's current page help you locate notices.
Version control is not clerical trivia. An evidence claim is tied to a particular text, dataset, analysis, and date. Saying "the paper showed" is incomplete if the paper has been materially corrected.
How to weigh an unreviewed study#
The absence of journal review should lower your confidence, but it does not determine the study's quality by itself. Appraisal still begins with the research question, design, population, comparator, and outcomes. A randomized design can reduce confounding; it cannot rescue poor randomization, high attrition, selective reporting, or a misleading outcome.
Check whether a protocol or registration predates the results and whether primary outcomes match. Examine sample size, missing data, effect estimates, confidence intervals, and absolute as well as relative effects. Look for multiplicity and analyses described as exploratory only after results appeared.
Funding and conflicts deserve attention without being treated as automatic disqualification. Data and code availability can permit deeper scrutiny, but a shared file is not proof that data are complete or correctly collected. Consistency with prior evidence matters, as does biological and clinical plausibility.
Comments from other researchers can be useful leads. They also vary in quality and may reflect partial reading or undisclosed conflicts. A critique should identify a reproducible issue, not merely express disbelief. Most important, avoid converting one unreviewed result into a clinical or policy conclusion. The appropriate weight depends on the total evidence, outcome urgency, potential harm of acting, and whether stronger studies are available.
Replication is cumulative, not a seal#
The National Academies distinguishes reproducibility, obtaining consistent computational results from the same data and procedures, from replicability, obtaining consistent results across new studies addressing the same question. Terminology varies across fields, so authors should name exactly what was repeated.
A replication is not a simple pass or fail. Results can differ because populations, measurements, implementation, analysis, or chance differ. Confidence intervals may be compatible even when one study is statistically significant and the other is not. A direct replication tests a narrower match; a conceptual replication tests whether the claim travels under changed conditions.
One unsuccessful replication can reveal a serious limitation without proving that every part of the original was false, and one successful replication strengthens evidence without proving the effect is universal or free of bias. Meta-analysis can combine studies, but synthesis inherits their design and reporting problems and must address heterogeneity. The durable unit of evidence is a body of transparent, critically appraised work, not the prestige of one paper.
Corrections, concerns, and retractions#
ICMJE recommends prompt correction of errors that affect the publication record. A correction can fix a label, author detail, numerical value, analysis, or conclusion. Its meaning depends on what changed. Some corrections leave the central result intact; others substantially alter interpretation.
An expression of concern alerts readers to a potentially serious issue while investigation continues. It is not a final finding, and a retraction signals that findings or reporting are unreliable enough that the article should not remain part of the literature in its original form. Retraction can follow misconduct or honest major error; the notice should explain the reason.
Post-publication discussion, data reanalysis, and later studies can identify issues that prepublication review missed. This continuing correction process is not evidence that science never knows anything. It is how a cumulative system updates claims in response to better information. So click through to the current journal record, check the notices, and never cite a retracted paper as valid evidence; when a correction matters, describe the corrected result rather than repeating the original headline.
A calibrated meaning of "peer reviewed"#
Peer review means that a journal applied its editorial process and obtained external assessment before its decision. It does not mean the reviewers agreed, saw all data, verified integrity, reproduced the analysis, or guaranteed clinical importance.
That calibrated definition preserves the value of review without outsourcing judgment to a label. The next questions remain: What was the design? Were methods appropriate and transparent? How large and precise was the effect? Are the outcomes important? Does the result fit other evidence? Has the record changed?
Publication is a milestone in scrutiny. It is not the end of it.
References#
- ICMJE Recommendations, updated January 2026
- ICMJE responsibilities in submission and peer review
- ICMJE guidance on preprints
- ICMJE corrections and version control
- Cochrane review of interventions to improve peer review
- Randomized study of reviewer error detection
- Empirical study of selective outcome reporting
- National Academies, Reproducibility and Replicability in Science
Questions and answers
Does peer review prove that a paper is correct?
No. It adds expert criticism and editorial assessment, but reviewers can miss errors and usually do not verify every underlying datum or repeat the study.
Who decides whether a manuscript is accepted?
The editor or editorial team decides. Reviewers provide advice, and their recommendations may disagree or be outweighed by other editorial considerations.
Is a preprint the same as a published journal article?
No. A preprint is public before journal peer review is complete. Server screening and version posting do not equal the journal's external review and editorial process.
Should a preprint be ignored?
Not automatically. Its design, protocol, estimates, uncertainty, reporting, and fit with prior evidence can be appraised, but its unreviewed status should be explicit and its conclusions weighted cautiously.
Does a correction mean the whole study is invalid?
Not necessarily. Corrections range from minor record changes to revisions that alter central findings. Readers need to examine what changed and use the current version. Retractions and expressions of concern have different meanings.