The rise of the registered report

Ok, give us a quick reminder of what a registered report actually is?

A registered report is an article format that is peer reviewed and registered before the start of data collection, or data analysis in case of submissions that involve secondary analyses of existing data. Journals provisionally commit to publish the study after its completion, irrespective of the results, provided that the authors have not substantially deviated from their registered protocol.

When authors submit a registered report to a journal, experts are asked to assess the importance of the research question and critically evaluate the protocol, giving researchers the opportunity to amend their plan and adopt the most rigorous research design.

Because the methods are peer-reviewed ahead of the start of the project, the format helps prevent publication bias and misuse of statistical analysis to support a particular conclusion.

Sharing research in this way is a pretty big shift – from a publisher’s perspective how are you finding this change?

Editors are keen to pursue the publication of studies performed as rigorously as possible on interesting research questions, and the format certainly fosters that. It’s always disheartening to read in reviewers’ reports on traditional manuscripts that a flaw in a study could have been avoided if the authors had been offered expert feedback before starting their investigation. Addressing the flaw after the fact can cost money and time.

When assessing registered reports, there is a shift in focus both for editors and reviewers as they need to move away from assessing the outcome and novelty of the work. They are in that case evaluating the importance of the research question and the soundness of the proposed methods; there is no result to look at yet.

There is evidence that this shift in focus is beneficial to reproducibility. A recent study by Nosek and colleagues evaluated side by side registered reports and traditional articles on similar topics and showed that the format could improve research quality and thus potentially improve the credibility of the published literature.

What do you think about the idea that registered reports could become the default way of publishing research?

The format is well suited to disciplines which lend themselves to testing hypotheses (confirmatory research).

More recently, several journals including Nature Methods, Nature Communications and Scientific Reports have opened the format to studies aimed at carrying out a comparison of the performance of established methods or tools. In that case, the experimental design and data analysis plan is peer-reviewed and registered prior to data collection but there is no intent to test a hypothesis.

Ultimately, I would like registered reports to be on the radar of researchers across disciplines as a format that may be suitable for them whenever they are setting out to test a hypothesis based on existing literature or comparing methods. However, the format in its current form is unlikely to be suited for research that is purely exploratory.

Give us your top tips for researchers thinking of publishing in this way…

I would strongly encourage researchers to consult the detailed information provided by the Center for Open Science ahead of laying down their research question and hypothesis, as well as getting familiar with journal specific guidelines for the format, as those may vary.

In terms of improving reproducibility and research integrity – do you think registered reports will be enough?

The format is one avenue to improve reproducibility and research integrity, but it is certainly not the only avenue and, as discussed, it may not be suitable in some cases.

Without going through the entire process of submitting a registered report to a journal, pre-registering your study plan on a site such as the Open Science Foundation, or on a recognised clinical trials registry when appropriate, provides rigour and accountability. However, registration within the context of the registered reports helps with both integrity and reproducibility, because the protocol is reviewed by other experts prior to registration.

It’s equally important to ensure that data and code linked to a published study are shared among researchers and detailed experimental protocols made available on an open platform.

At Nature Communications, we also believe that making the publication process transparent is key to ensure accountability of researchers and reviewers and thus, the exchange occurring between reviewers and researchers during the peer-review process is published alongside all our articles.

Find out more about Registered Reports and our pilot Registered Reports Funding Partnership with a consortium of 12 journals here