The impact of peer review on the contribution potential of scientific papers

Akira Matsui; Emily Chen; Yunwen Wang; Emilio Ferrara

doi:10.7717/peerj.11999

The impact of peer review on the contribution potential of scientific papers

PeerJ. 2021 Sep 15:9:e11999. doi: 10.7717/peerj.11999. eCollection 2021.

Authors

Akira Matsui^#^{1

2}, Emily Chen^#^{1

2}, Yunwen Wang³, Emilio Ferrara^{1

2

3}

Affiliations

¹ Department of Computer Science, University of Southern California, Los Angeles, California, United States.
² Information Sciences Institute, University of Southern California, Marina del Rey, California, United States.
³ Annenberg School for Communication and Journalism, University of Southern California, Los Angeles, California, United States.

^# Contributed equally.

Abstract

The peer-reviewing process has long been regarded as an indispensable tool in ensuring the quality of a scientific publication. While previous studies have tried to understand the process as a whole, not much effort has been devoted to investigating the determinants and impacts of the content of the peer review itself. This study leverages open data from nearly 5,000 PeerJ publications that were eventually accepted. Using sentiment analysis, Latent Dirichlet Allocation (LDA) topic modeling, mixed linear regression models, and logit regression models, we examine how the peer-reviewing process influences the acceptance timeline and contribution potential of manuscripts, and what modifications were typically made to manuscripts prior to publication. In an open review paradigm, our findings indicate that peer reviewers' choice to reveal their names in lieu of remaining anonymous may be associated with more positive sentiment in their review, implying possible social pressure from name association. We also conduct a taxonomy of the manuscript modifications during a revision, studying the words added in response to peer reviewer feedback. This study provides insights into the content of peer reviews and the subsequent modifications authors make to their manuscripts.

Keywords: Peer review system; Science of science; Science policy.

Grants and funding

This work was supported by DARPA (grant no. D16AP00115 and contract no. W911NF-17-C-0094). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.