Reproducible research practices, transparency, and open access data in the biomedical literature, 2015-2017

PLoS Biol. 2018 Nov 20;16(11):e2006930. doi: 10.1371/journal.pbio.2006930. eCollection 2018 Nov.

Abstract

Currently, there is a growing interest in ensuring the transparency and reproducibility of the published scientific literature. According to a previous evaluation of 441 biomedical journals articles published in 2000-2014, the biomedical literature largely lacked transparency in important dimensions. Here, we surveyed a random sample of 149 biomedical articles published between 2015 and 2017 and determined the proportion reporting sources of public and/or private funding and conflicts of interests, sharing protocols and raw data, and undergoing rigorous independent replication and reproducibility checks. We also investigated what can be learned about reproducibility and transparency indicators from open access data provided on PubMed. The majority of the 149 studies disclosed some information regarding funding (103, 69.1% [95% confidence interval, 61.0% to 76.3%]) or conflicts of interest (97, 65.1% [56.8% to 72.6%]). Among the 104 articles with empirical data in which protocols or data sharing would be pertinent, 19 (18.3% [11.6% to 27.3%]) discussed publicly available data; only one (1.0% [0.1% to 6.0%]) included a link to a full study protocol. Among the 97 articles in which replication in studies with different data would be pertinent, there were five replication efforts (5.2% [1.9% to 12.2%]). Although clinical trial identification numbers and funding details were often provided on PubMed, only two of the articles without a full text article in PubMed Central that discussed publicly available data at the full text level also contained information related to data sharing on PubMed; none had a conflicts of interest statement on PubMed. Our evaluation suggests that although there have been improvements over the last few years in certain key indicators of reproducibility and transparency, opportunities exist to improve reproducible research practices across the biomedical literature and to make features related to reproducibility more readily visible in PubMed.

Publication types

  • Research Support, N.I.H., Extramural
  • Research Support, Non-U.S. Gov't

MeSH terms

  • Access to Information / ethics
  • Biomedical Research / economics*
  • Biomedical Research / ethics*
  • Conflict of Interest / economics
  • Disclosure / ethics
  • Disclosure / standards
  • Humans
  • Information Dissemination / ethics
  • Information Dissemination / methods
  • Publications / ethics
  • Reproducibility of Results

Grant support

National Institute on Drug Abuse, National Institutes of Health https://projectreporter.nih.gov/project_info_description.cfm?aid=9583616&icde=41489254 (grant number HHSN271201700041C). Received by KWB and JPAI. The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Laura and John Arnold Foundation. Received by the Meta-Research Innovation Center at Stanford (METRICS). The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.