Objective: To investigate whether discrepancies in trials of use of bone marrow stem cells in patients with heart disease account for the variation in reported effect size in improvement of left ventricular function.
Design: Identification and counting of factual discrepancies in trial reports, and sample size weighted regression against therapeutic effect size. Meta-analysis of trials that provided sufficient information.
Data sources: PubMed and Embase from inception to April 2013.
Eligibility for selecting studies: Randomised controlled trials evaluating the effect of autologous bone marrow stem cells for heart disease on mean left ventricular ejection fraction.
Results: There were over 600 discrepancies in 133 reports from 49 trials. There was a significant association between the number of discrepancies and the reported increment in EF with bone marrow stem cell therapy (Spearman's r=0.4, P=0.005). Trials with no discrepancies were a small minority (five trials) and showed a mean EF effect size of -0.4%. The 24 trials with 1-10 discrepancies showed a mean effect size of 2.1%. The 12 with 11-20 discrepancies showed a mean effect of size 3.0%. The three with 21-30 discrepancies showed a mean effect size of 5.7%. The high discrepancy group, comprising five trials with over 30 discrepancies each, showed a mean effect size of 7.7%.
Conclusions: Avoiding discrepancies is difficult but is important because discrepancy count is related to effect size. The mechanism is unknown but should be explored in the design of future trials because in the five trials without discrepancies the effect of bone marrow stem cell therapy on ejection fraction is zero.