How to peer review Registered Reports

How does reviewing a Registered Report differ from traditional peer review?

Reviewing Registered Reports differs from the traditional peer review process in two key ways:

1. Two stages of peer review are carried out, before and after research is conducted. In the first stage reviewers are asked to assess a study proposal, before the proposed research has been carried out. In the second stage they are asked to consider the completed study, including all results and interpretations.

2. The factors that reviewers are asked to assess are different from those for a standard research article. Where a traditional peer review process might ask you to assess the novelty or significance of results reported in a manuscript, Registered Reports facilitate a “results-blind” review process which helps reduce publication bias against null or inconclusive results.

Reviewing both stages of a Registered Report

Where possible, we try to ask the same group of reviewers to assess each stage of a Registered Report. If you agree to review an initial Registered Report Protocol, we may also contact you to ask you to review the final Registered Report once it has been completed.

Journal-specific expectations

The guidance below is intended to give a general overview of the typical peer review process for a Registered Report. However, individual journals may have specific requirements that are relevant to their particular research domains. If you have been asked to peer review a Registered Report by a Cambridge journal, please make sure to check that journal’s guidance for peer reviewers for any additional expectations. This can be found in the “Journal information” section of the journal’s home page, under “Peer review information”.

Stage 1: The Registered Report Protocol (RRP)

In the first stage of peer review, you will be asked to assess a study proposal, or “Registered Report Protocol”, before any research data have been collected or analysed. Registered Report Protocols typically consist only of an Introduction and Methods section, which should include details of all proposed analyses. In some cases they may also include pilot data collected to establish proof of concept for a study.

The purpose of peer review at this stage is to assess whether the RRP has proposed a valuable research question, as well as an appropriate study design to address it. You will be asked to consider factors such as:

The importance of the research question being asked
The logic, rationale, and plausibility of the proposed hypotheses
The soundness and feasibility of the proposed methodology and analysis pipeline
Whether the proposed research offers an appropriate test of the hypotheses
Whether the proposed methods are sufficiently detailed
Whether the authors have specified sufficient controls and quality checks to ensure the results obtained can test the hypotheses

If a Registered Report Protocol passes this round of peer review, it will be given “in-principle acceptance” from the journal, and the authors will be asked to go and complete the research they have proposed. After this they will submit their completed Registered Report, which will undergo a second round of peer review.

Stage 2: The completed Registered Report

In the second round of peer review, you will be asked to assess the full study in the completed Registered Report. This completed manuscript should consist of the original Introduction and Methods sections from the RRP, as well as Results and Discussion sections detailing the results, interpretations, and conclusions of all analyses that have been carried out.

At this stage completed Registered Reports resemble a typical research article, but are still reviewed based on different criteria to a traditional peer review. Most importantly, editorial decisions about completed Registered Reports will not be based on the perceived importance, novelty, or conclusiveness of the results. The purpose of peer review at this stage is to assess whether the authors have followed their proposed study design, and clearly explained and justified any deviations from what they proposed in their original RRP. You will be asked to compare the completed Registered Report to the initial Registered Report Protocol and ensure that:

The research question, rationale, and hypotheses have not changed from the RRP
The authors have precisely followed the experimental procedures detailed in their RRP
Any departures from the proposed experimental procedures have been explained and justified
Any post hoc analyses that were not proposed in the RRP are clearly labelled, justified, methodologically sound, and informative
The authors’ conclusions are justified given the nature and quality of the data and analyses

If you wish, at this stage you may also suggest additional post hoc tests for the authors to carry out on their data – although authors are not obliged to carry out these tests unless they are necessary to satisfy other review criteria.