Skip to main page content
Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
, 6, 588

What Is Open Peer Review? A Systematic Review


What Is Open Peer Review? A Systematic Review

Tony Ross-Hellauer. F1000Res.


Background: "Open peer review" (OPR), despite being a major pillar of Open Science, has neither a standardized definition nor an agreed schema of its features and implementations. The literature reflects this, with numerous overlapping and contradictory definitions. While for some the term refers to peer review where the identities of both author and reviewer are disclosed to each other, for others it signifies systems where reviewer reports are published alongside articles. For others it signifies both of these conditions, and for yet others it describes systems where not only "invited experts" are able to comment. For still others, it includes a variety of combinations of these and other novel methods. Methods: Recognising the absence of a consensus view on what open peer review is, this article undertakes a systematic review of definitions of "open peer review" or "open review", to create a corpus of 122 definitions. These definitions are systematically analysed to build a coherent typology of the various innovations in peer review signified by the term, and hence provide the precise technical definition currently lacking. Results: This quantifiable data yields rich information on the range and extent of differing definitions over time and by broad subject area. Quantifying definitions in this way allows us to accurately portray exactly how ambiguously the phrase "open peer review" has been used thus far, for the literature offers 22 distinct configurations of seven traits, effectively meaning that there are 22 different definitions of OPR in the literature reviewed. Conclusions: I propose a pragmatic definition of open peer review as an umbrella term for a number of overlapping ways that peer review models can be adapted in line with the aims of Open Science, including making reviewer and author identities open, publishing review reports and enabling greater participation in the peer review process.

Keywords: Open Science; open peer review; publishing; research evaluation; scholarly communication.

Conflict of interest statement

Competing interests: This work was conducted as part of the OpenAIRE2020 project, an EC-funded initiative to implement and monitor Open Access and Open Science policies in Europe and beyond.


Figure 1.
Figure 1.. Definitions of OPR in the literature by year.
Figure 2.
Figure 2.. Breakdown of OPR definitions by source.
Figure 3.
Figure 3.. Breakdown of OPR definitions by disciplinary scope.
Figure 4.
Figure 4.. Breakdown of OPR definitions by type of material being reviewed.
Figure 5.
Figure 5.. Distribution of OPR traits amongst definitions.
Figure 6.
Figure 6.. Prevalence of traits (as percentage) within definitions by disciplinary focus of definition.
Figure 7.
Figure 7.. Unique configurations of OPR traits within definitions.
Figure 8.
Figure 8.. Five schools of thought in Open Science (CC BY-NC, Fecher & Friesike, 2013).

Similar articles

See all similar articles

Cited by 19 PubMed Central articles

See all "Cited by" articles


    1. Armstrong JS: Barriers to Scientific Contributions: The Authors Formula. Behav Brain Sci. Cambridge University Press (CUP). 1982;5(02):197– 199 10.1017/s0140525x00011201 - DOI
    1. Armstrong JS: Peer Review for Journals: Evidence on Quality Control Fairness, and Innovation. Sci Eng Ethics. Springer Nature. 1997;3(1):63–84. 10.1007/s11948-997-0017-3 - DOI
    1. Bardy AH: Bias in reporting clinical trials. Br J Clin Pharmacol. Wiley-Blackwell. 1998;46(2):147–50. 10.1046/j.1365-2125.1998.00759.x - DOI - PMC - PubMed
    1. Bloom T: Referee Report For: What is open peer review? A systematic review [version 1; referees: 1 approved, 3 approved with reservations]. F1000Res. 2017;6:588 10.5256/f1000research.12273.r22301 - DOI - PMC - PubMed
    1. Boldt A: Extending ArXiv.Org to Achieve Open Peer Review and Publishing. J Scholarly Publ. University of Toronto Press Inc. (UTPress), 2011;42(2):238–42. 10.3138/jsp.42.2.238 - DOI

Grant support

This work is funded by the European Commission H2020 project OpenAIRE2020 (Grant agreement: 643410, Call: H2020-EINFRA-2014-1)

LinkOut - more resources