Skip to main page content
Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
, 20 (12), e11924

Nonpublication Rates and Characteristics of Registered Randomized Clinical Trials in Digital Health: Cross-Sectional Analysis


Nonpublication Rates and Characteristics of Registered Randomized Clinical Trials in Digital Health: Cross-Sectional Analysis

Mustafa Al-Durra et al. J Med Internet Res.


Background: Clinical trials are key to advancing evidence-based medical research. The medical research literature has identified the impact of publication bias in clinical trials. Selective publication for positive outcomes or nonpublication of negative results could misdirect subsequent research and result in literature reviews leaning toward positive outcomes. Digital health trials face specific challenges, including a high attrition rate, usability issues, and insufficient formative research. These challenges may contribute to nonpublication of the trial results. To our knowledge, no study has thus far reported the nonpublication rates of digital health trials.

Objective: The primary research objective was to evaluate the nonpublication rate of digital health randomized clinical trials registered in Our secondary research objective was to determine whether industry funding contributes to nonpublication of digital health trials.

Methods: To identify digital health trials, a list of 47 search terms was developed through an iterative process and applied to the "Title," "Interventions," and "Outcome Measures" fields of registered trials with completion dates between April 1, 2010, and April 1, 2013. The search was based on the full dataset exported from the database, with 265,657 trials entries downloaded on February 10, 2018, to allow publication of studies within 5 years of trial completion. We identified publications related to the results of the trials through a comprehensive approach that included an automated and manual publication-identification process.

Results: In total, 6717 articles matched the a priori search terms, of which 803 trials matched our latest completion date criteria. After screening, 556 trials were included in this study. We found that 150 (27%) of all included trials remained unpublished 5 years after their completion date. In bivariate analyses, we observed statistically significant differences in trial characteristics between published and unpublished trials in terms of the intervention target condition, country, trial size, trial phases, recruitment, and prospective trial registration. In multivariate analyses, differences in trial characteristics between published and unpublished trials remained statistically significant for the intervention target condition, country, trial size, trial phases, and recruitment; the odds of publication for non-US-based trials were significant, and these trials were 3.3 (95% CI 1.845-5.964) times more likely to be published than US-based trials. We observed a trend of 1.5 times higher nonpublication rates for industry-funded trials. However, the trend was not statistically significant.

Conclusions: In the domain of digital health, 27% of registered clinical trials results are unpublished, which is lower than nonpublication rates in other fields. There are substantial differences in nonpublication rates between trials funded by industry and nonindustry sponsors. Further research is required to define the determinants and reasons for nonpublication and, more importantly, to articulate the impact and risk of publication bias in the field of digital health trials.

Keywords: clinical protocols; clinical trial; eHealth; mHealth; mobile health; publication bias; publications; randomized controlled trial; registries; telehealth; telemedicine.

Conflict of interest statement

Conflicts of Interest: GE is the editor-in-chief of the Journal of Medical Internet Research (and publisher at JMIR Publications) but was not involved in the peer-review or decision-making process for this paper. The associate editor handling this manuscript and the reviewers were blinded and not aware of the co-authorship of GE. As owner of JMIR Publications, GE may benefit from increased publication rates of digital health trials. The other authors declare no conflicts of interests.


Figure 1
Figure 1
Trials included from the search results.
Figure 2
Figure 2
Results of the publication-identification process. *NCT: National Clinical Trial.
Figure 3
Figure 3
Time to publication of registered clinical trials in digital health.

Similar articles

See all similar articles

Cited by 1 article


    1. Rosenthal R. The file drawer problem and tolerance for null results. Psychological Bulletin. 1979;86(3):638–641. doi: 10.1037/0033-2909.86.3.638. - DOI
    1. Dickersin K. The existence of publication bias and risk factors for its occurrence. JAMA. 1990 Mar 09;263(10):1385–9. - PubMed
    1. Harrison BA, Mayo-Wilson E. Trial Registration. Research on Social Work Practice. 2013 Nov 19;24(3):372–376. doi: 10.1177/1049731513512374. - DOI
    1. Wager E, Williams P, Project Overcome failure to Publish nEgative fiNdings Consortium “Hardly worth the effort”? Medical journals' policies and their editors' and publishers' views on trial registration and publication bias: quantitative and qualitative study. BMJ. 2013 Sep 06;347:f5248. doi: 10.1136/bmj.f5248. - DOI - PMC - PubMed
    1. Simes RJ. Publication bias: the case for an international registry of clinical trials. J Clin Oncol. 1986 Oct;4(10):1529–41. doi: 10.1200/JCO.1986.4.10.1529. - DOI - PubMed

LinkOut - more resources