Skip to main page content
Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2015 Aug 26;10(8):e0136088.
doi: 10.1371/journal.pone.0136088. eCollection 2015.

Replication, Communication, and the Population Dynamics of Scientific Discovery

Affiliations
Free PMC article

Replication, Communication, and the Population Dynamics of Scientific Discovery

Richard McElreath et al. PLoS One. .
Free PMC article

Abstract

Many published research results are false (Ioannidis, 2005), and controversy continues over the roles of replication and publication policy in improving the reliability of research. Addressing these problems is frustrated by the lack of a formal framework that jointly represents hypothesis formation, replication, publication bias, and variation in research quality. We develop a mathematical model of scientific discovery that combines all of these elements. This model provides both a dynamic model of research as well as a formal framework for reasoning about the normative structure of science. We show that replication may serve as a ratchet that gradually separates true hypotheses from false, but the same factors that make initial findings unreliable also make replications unreliable. The most important factors in improving the reliability of research are the rate of false positives and the base rate of true hypotheses, and we offer suggestions for addressing each. Our results also bring clarity to verbal debates about the communication of research. Surprisingly, publication bias is not always an obstacle, but instead may have positive impacts-suppression of negative novel findings is often beneficial. We also find that communication of negative replications may aid true discovery even when attempts to replicate have diminished power. The model speaks constructively to ongoing debates about the design and conduct of science, focusing analysis and discussion on precise, internally consistent models, as well as highlighting the importance of population dynamics.

Conflict of interest statement

Competing Interests: The authors have declared that no competing interests exist.

Figures

Fig 1
Fig 1. Population dynamics of replication.
Fig 2
Fig 2. Effects of base rate, replication, power, false-positives, and communication on the probability that an hypothesis with a given tally is true.
The two clusters illustrate difference scenarios. The blue trends, each labeled with its tally value, show precision as it varies by the parameter on each horizontal axis. The numbers indicate the tally of a curve. Dashed curves are tallies of an even number. The vertical hairlines show the parameter values held constant across panels within the same cluster.
Fig 3
Fig 3. Replication and communication as epistemological chromatography.
Precision is indicated in blue, sensitivity in orange, and specificity in gray.
Fig 4
Fig 4. Targeted replication effort.
In all three plots, tallies marked for targeted replication are shown by the shaded region. Precision is indicated in blue, sensitivity in orange, and specificity in gray. Baseline parameters set to b = 0.001, α = 0.05, r = 0.1, r T = 0.5, c N− = 0, c R− = c R+ = 1. Dashed curves display steady-state without targeted replication, r T = 0. (a) High power setting, 1 − β = 0.8. (b) Low power setting, 1 − β = 0.6. (c) Low power, 1 − β = 0.6, and including tally s = 0 in the target.
Fig 5
Fig 5. Differential power and replication dynamics.
Precision is indicated in blue, sensitivity in orange, and specificity in gray. (a) Low power initial studies (1 − β = 0.6, α = 0.2) but high power replications (1 − β R = 0.8, α R = 0.05). (b) High power initial studies (1 − β = 0.8, α = 0.05) but low power replications (1 − β R = 0.5, α R = 0.05). (c) and (d) as in (a) and (b), respectively, but only 10% of negative replications are communicated.

Similar articles

See all similar articles

Cited by 22 articles

See all "Cited by" articles

References

    1. Ioannidis JPA. Why Most Published Research Findings Are False. PLoS Med. 2005. August;2(8):e124 Available from: 10.1371/journal.pmed.0020124 - DOI - PMC - PubMed
    1. Makel MC, Plucker JA, Hegarty B. Replications in Psychology Research How Often Do They Really Occur? Perspectives on Psychological Science. 2012;7(6):537–542. Available from: http://pps.sagepub.com/content/7/6/537 10.1177/1745691612460688 - DOI - PubMed
    1. Franco A, Malhotra N, Simonovits G. Publication bias in the social sciences: Unlocking the file drawer. Science. 2014;345:1502–1505. Available from: http://www.sciencemag.org/content/345/6203/1502 10.1126/science.1255484 - DOI - PubMed
    1. Schmidt S. Shall we really do it again? The powerful concept of replication is neglected in the social sciences. Review of General Psychology. 2009;13(2):90–100. 10.1037/a0015108 - DOI
    1. Begley CG, Ellis LM. Drug development: Raise standards for preclinical cancer research. Nature. 2012;483:531–533. Available from: http://www.nature.com/nature/journal/v483/n7391/full/483531a.html 10.1038/483531a - DOI - PubMed

Publication types

Grant support

The Division of Social Sciences Dean’s Office at the University of California Davis provided financial support. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Feedback