Replication, Communication, and the Population Dynamics of Scientific Discovery

PLoS One. 2015 Aug 26;10(8):e0136088. doi: 10.1371/journal.pone.0136088. eCollection 2015.

Abstract

Many published research results are false (Ioannidis, 2005), and controversy continues over the roles of replication and publication policy in improving the reliability of research. Addressing these problems is frustrated by the lack of a formal framework that jointly represents hypothesis formation, replication, publication bias, and variation in research quality. We develop a mathematical model of scientific discovery that combines all of these elements. This model provides both a dynamic model of research as well as a formal framework for reasoning about the normative structure of science. We show that replication may serve as a ratchet that gradually separates true hypotheses from false, but the same factors that make initial findings unreliable also make replications unreliable. The most important factors in improving the reliability of research are the rate of false positives and the base rate of true hypotheses, and we offer suggestions for addressing each. Our results also bring clarity to verbal debates about the communication of research. Surprisingly, publication bias is not always an obstacle, but instead may have positive impacts-suppression of negative novel findings is often beneficial. We also find that communication of negative replications may aid true discovery even when attempts to replicate have diminished power. The model speaks constructively to ongoing debates about the design and conduct of science, focusing analysis and discussion on precise, internally consistent models, as well as highlighting the importance of population dynamics.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Biomedical Research / standards*
  • Communication*
  • Humans
  • Models, Theoretical
  • Population Dynamics*
  • Publication Bias
  • Reproducibility of Results
  • Research Design / standards*
  • Science

Grants and funding

The Division of Social Sciences Dean’s Office at the University of California Davis provided financial support. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.