Blog

As Editorial Assistants, our peer reviewers will often say to us “I won’t review the Study Protocol, but I will review the results”, which made us wonder why Study Protocols are disregarded by some? In this blog post, Charlie Vickers, Senior Editorial Assistant, explains why Study Protocols are important – not only for science in general, but as part of the AAS Open Research model too.

An overview of protocols

Put simply, a Study Protocol is the introduction and methods section of a standard research article (or other article types, such as a systematic review), without having the results and discussion section included. In most cases, this is because the authors have not yet begun collecting their data, but as with most things there are exceptions, such as if cohort data is to be analysed as part of the research.

Some Protocols may have some data associated (which is accessible, owing to our open data policy), for example a preliminary study to determine whether the method works and is effective. Peer reviewers can then see the benefits of the work to be undertaken, as well as the validation of the study design itself.

One of the main advantages of publishing a Study Protocol is that the methods can be peer reviewed before undertaking the work described. The reviewers will evaluate the outlined Protocol, including any statistical analyses the authors plan to perform on the collected data. Peer review at this stage is also beneficial when it comes to publishing the completed research, as this allows any potential problems with the methods to be outlined, so that reviewers of the resulting research article are unlikely to ask you to perform more studies before giving their approval.

Science and the wider community can also benefit from the publication of Study Protocols, as it prevents studies being altered following the collection of the data. The ‘reproducibility crisis’ as it has come to be known, has been recognised for several years now and is believed to be, in part, caused by the publication of altered research to give more favourable results.

Publishing a Study Protocol doesn’t mean that the results have to be published in the same journal, or even be published at all. This study within a protocol, published on the F1000Research powered platform HRB Open Research, was designed and ready to be implemented before the wider trial was cancelled. However, the methods were still published, as they are still a useful contribution to the academic literature, allowing the work to be used and cited by other groups with similar ideas in the future.

Registered Reports

This all ties nicely into what we think is an important aspect of publishing Study Protocols – Registered Reports, which are slowly becoming more common as more people understand the concept. The problem of publication bias has long been understood and Registered Reports are a relatively new way of making research available to overcome this problem. Examples of publication bias include only publishing positive results or altering the hypothesis of a study retrospectively to fit with the results produced.

In order to achieve this, the Study Protocol (often called the Stage 1) is submitted to the journal before the research is conducted, sent for peer review and then eventually accepted. At this stage, the research can be undertaken, ensuring that the methods and statistical analysis are followed as planned and any results are published, even if they give a negative or null result. If the authors make any changes to the methods, these need to be justified when publishing the research article (known as Stage 2, containing the extra results and discussion).

Different journals currently offer a few different methods to publish Registered Reports (shown under the Participating Journals tab). At AAS Open Research we take the approach of publishing and peer reviewing the Stage 1 of a Registered Report, before the Methods are implemented and the data collected. This can then be written up and submitted to us as the Stage 2 of the Registered Report, the research article. Our usual editorial checks are then undertaken before publication, where we invite the same reviewers to assess the research article. We believe this approach is best for transparency, making sure that readers can trust the methods have been followed as described in the initial Protocol (unless a justification has been made) preventing the analysis and outcomes being manipulated to get the results the researcher wants.

The AAS Open Research approach to Registered Reports means that both stages of the Registered Report are available as citable publications, as well as the peer review reports and any changes, either between versions or stages. Other journals may have different approaches, such as sending the Study Protocol out for review but not publishing the Protocol until the research article containing the full results is published also.

In summary

To us at AAS Open Research, there really doesn’t seem to be any reason why publishing Study Protocols, either as standalone articles or as part of a Registered Report, won’t become the norm in the coming years. They increase transparency and allow methods to be properly assessed before proceeding with the research, so why wouldn’t you want to help this process by reviewing one? 

If you are interested in reviewing for us, and would like to receive credit for your hard work, please get in touch with the AAS Open Research Editorial Team.

You can also download our free guide for authors on what to cover in your Stage 1 Study Protocol. This template explains exactly what you need to include in every section of your article, from the Abstract through to the Data Availability Statement, and more.