Reproducibility of preclinical animal research improves with heterogeneity of study samples

PLoS Biol. 2018 Feb 22;16(2):e2003693. doi: 10.1371/journal.pbio.2003693. eCollection 2018 Feb.

Abstract

Single-laboratory studies conducted under highly standardized conditions are the gold standard in preclinical animal research. Using simulations based on 440 preclinical studies across 13 different interventions in animal models of stroke, myocardial infarction, and breast cancer, we compared the accuracy of effect size estimates between single-laboratory and multi-laboratory study designs. Single-laboratory studies generally failed to predict effect size accurately, and larger sample sizes rendered effect size estimates even less accurate. By contrast, multi-laboratory designs including as few as 2 to 4 laboratories increased coverage probability by up to 42 percentage points without a need for larger sample sizes. These findings demonstrate that within-study standardization is a major cause of poor reproducibility. More representative study samples are required to improve the external validity and reproducibility of preclinical animal research and to prevent wasting animals and resources for inconclusive research.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Animal Experimentation / standards*
  • Animals
  • Disease Models, Animal
  • Laboratories / organization & administration*
  • Probability
  • Reproducibility of Results
  • Research Design / standards
  • Sample Size

Grant support

European Research Council ERC (grant number 322576). Swiss Food Safety and Veterinary Office FSVO (grant number 2.13.01). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.