Skip to main page content
Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
, 10 (1), 322

The Effect of Publishing Peer Review Reports on Referee Behavior in Five Scholarly Journals

Affiliations

The Effect of Publishing Peer Review Reports on Referee Behavior in Five Scholarly Journals

Giangiacomo Bravo et al. Nat Commun.

Abstract

To increase transparency in science, some scholarly journals are publishing peer review reports. But it is unclear how this practice affects the peer review process. Here, we examine the effect of publishing peer review reports on referee behavior in five scholarly journals involved in a pilot study at Elsevier. By considering 9,220 submissions and 18,525 reviews from 2010 to 2017, we measured changes both before and during the pilot and found that publishing reports did not significantly compromise referees' willingness to review, recommendations, or turn-around times. Younger and non-academic scholars were more willing to accept to review and provided more positive and objective recommendations. Male referees tended to write more constructive reports during the pilot. Only 8.1% of referees agreed to reveal their identity in the published report. These findings suggest that open peer review does not compromise the process, at least when referees are able to protect their anonymity.

Conflict of interest statement

The authors declare no competing interests.

Figures

Fig. 1
Fig. 1
Number of monthly submissions in the pilot journals
Fig. 2
Fig. 2
Proportion of referees who accepted the editors’ invitation by journal. Thicker curves show smoothed fitting of the data (Loess) for each journal. The last 6 months were removed from the figure due to few observations
Fig. 3
Fig. 3
Gender and status distribution of referees by review condition. Error bars represent 95% CI obtained via bootstrap (1000 samples)
Fig. 4
Fig. 4
Proportion of recommendations by review condition and name disclosure. Error bars represent 95% CI obtained via bootstrap (1000 samples)
Fig. 5
Fig. 5
Distribution of polarity and subjectivity in the report text before and during the pilot. Note that for polarity, the interval was [−1, 1], larger numbers indicating a more positive tone, while for subjectivity the interval was [0, 1], higher numbers indicating more subjective reports

Similar articles

See all similar articles

Cited by 4 PubMed Central articles

References

    1. Walker R, Rocha da Silva P. Emerging trends in peer reviewa survey. Front. Neurosci. 2015;9:169. doi: 10.3389/fnins.2015.00169. - DOI - PMC - PubMed
    1. Teele DL, Thelen K. Gender in the journals: publication patterns in political science. PS Polit. Sci. Polit. 2017;50:433–447. doi: 10.1017/S1049096516002985. - DOI
    1. Siler K, Lee K, Bero L. Measuring the effectiveness of scientific gatekeeping. Proc. Natl Acad. Sci. USA. 2015;112:360–365. doi: 10.1073/pnas.1418218112. - DOI - PMC - PubMed
    1. Strang D, Siler K. Revising as reframing: original submissions versus published papers in administrative science quarterly, 2005 to 2009. Sociol. Theor. 2015;33:71–96. doi: 10.1177/0735275115572152. - DOI
    1. Balietti S, Goldstone RL, Helbing D. Peer review and competition in the art exhibition game. Proc. Natl. Acad. Sci. USA. 2016;113:8414–8419. doi: 10.1073/pnas.1603723113. - DOI - PMC - PubMed

Publication types

Feedback