How Preregistration and Registered Reports Curb Bad Science

One question separates a study you can lean on from one you should hold at arm's length: did the analysis plan come before the data, or after? Preregistration and Registered Reports exist to make that question answerable. Preregistration puts a study's hypotheses and analysis plan into a public archive with a timestamp before any data are collected. Registered Reports go a step further and have a journal accept the study on the strength of its methods, before a single result exists.

Key points#

• Preregistration records what a study intends to test, and when, so confirmatory claims can be told apart from after-the-fact ones.
• Registered Reports move peer review to before the results, so a journal commits to publishing whatever the data show.
• Both tools target HARKing, p-hacking, and publication bias, the habits that let chance findings pass as confirmed predictions.
• For a reader, the shortcut is simple: check whether the registered plan matches the analyses the paper actually reports.

Why the order of plan and results matters#

Two everyday research habits inflate the published record without anyone intending fraud. The first is HARKing, short for hypothesizing after the results are known. A team runs an analysis, spots an unexpected pattern, and writes the paper as though that pattern had been the prediction from the start. The second is p-hacking: testing many outcomes, subgroups, or model specifications and reporting only the version that clears the bar for statistical significance. Both can feel like careful, thorough work to the people doing them, which is precisely why they spread.

Publication bias then compounds the damage. Journals have long favored clean, positive, surprising findings, so studies that turned up nothing often stayed unpublished. Readers see the hits and never tally the misses, and any single striking result looks sturdier than the full body of attempts would suggest.

How much does this matter in practice? A large demonstration came in 2015, when the Open Science Collaboration set out to repeat 100 studies from three psychology journals, publishing the effort in Science. Ninety-seven percent of the original studies had reported statistically significant results, but only 36 percent of the repeats did, and the effects that did replicate were on average roughly half the size of the originals. That shortfall, often called the replication crisis, is the gap these two tools were built to narrow.

Preregistration: a timestamped plan#

Preregistration is the lighter of the two. Before touching any data, researchers write down their hypotheses, their target sample size and stopping rule, their primary and secondary outcomes, and the specific statistical tests they will run, then file that document in a public registry that stamps the date. Clinical trials have required this for years through registries such as ClinicalTrials.gov, and the practice now reaches into observational and laboratory research as well.

The aim is not to outlaw discovery. A preregistered study can still chase an interesting surprise in the data; it simply has to label that work as exploratory rather than confirmatory. That one label does most of the job. A confirmatory test named in advance keeps the statistical meaning it was designed to carry. An exploratory test, picked after the data are in view, is a lead worth following later, not a settled conclusion. Preregistration draws a visible boundary between the two, so a reader does not have to take the author's word for which is which.

Its weakness is enforcement. A filed plan is only a plan; nothing forces an author or a journal to honor it, and reviews of published work have found papers that drift from their registered protocols without flagging the change. Preregistration hands you a document to check against. It does not promise that anyone checked.

Registered Reports: review before the results#

Registered Reports close that loophole by shifting peer review to a point before the results exist. In the format, which the Center for Open Science coordinates and tracks, review happens in two stages. At the first stage, authors submit only the introduction, methods, and analysis plan. Editors and reviewers judge whether the question is worth answering and whether the design can actually answer it, and they can require changes while changes are still cheap to make. A protocol that clears this bar receives what is called in-principle acceptance.

In-principle acceptance is the structural heart of the idea. It means the journal agrees in advance to publish the finished study no matter how the results land, provided the authors follow the approved protocol and meet basic quality standards. At the second stage, once the data are collected, authors add the results and discussion and keep any unplanned analyses in a clearly marked exploratory section. The acceptance cannot be withdrawn simply because the findings came back null or unremarkable.

That structure pulls the incentive to p-hack or HARK out of the process, because a positive result no longer earns a publication that a negative result would be refused. It also puts the most useful criticism up front, at the moment when a reviewer's suggestion can still improve the study rather than merely grade it. More than 300 journals now offer the format.

Does the format deliver better studies?#

The early evidence points in a hopeful direction. A 2021 study in Nature Human Behaviour by Soderberg and colleagues asked 353 researchers to blindly rate 29 published Registered Reports against 57 comparison papers across 19 criteria. The Registered Reports came out ahead overall, with their clearest advantages in rigor of methodology and rigor of analysis, and no cost to novelty or creativity. Committing to a fixed plan, in other words, raised quality without dulling the ideas.

How to read a paper with this lens#

You do not need to run any of this yourself to benefit from it. When you pick up a study, look for a few signals. Is there a link to a preregistration or a Registered Report badge? Do the confirmatory analyses in the paper match what the plan described, and are the after-the-fact analyses labeled as exploratory? A study whose plan can be shown to predate its results has earned more of your trust than one where the sequence is impossible to reconstruct. Plan before results, or results before plan, is one of the sharpest questions you can bring to any claim.