Generalizability of findings from systematic reviews and meta-analyses in the Leading General Medical Journals

J Rehabil Med. 2020 Mar 18;52(3):jrm00031. doi: 10.2340/16501977-2659.

Abstract

Objective: To assess how items relevant for the assessment of the generalizability of findings from randomized controlled trials were recorded in systematic reviews published in leading general medical journals.

Methods: All systematic reviews and meta-analyses published in the Annals of Internal Medicine, BMJ, JAMA (The Journal of the American Medical Association) and Lancet from 1 January 2016 to 28 February 2019 were searched via PubMed. Reporting of the characteristics of randomized controlled trials in the systematic reviews was documented by the benchmarking method.

Results: A total of 115 systematic reviews were found. Of these, 71% included pharmacological interventions, 35% included other conservative treatments, 13% included surgical interventions, and 0% included rehabilitation interventions. None of the systematic reviews assessed patient selection, 35% reported disorder-specific clinical features, 25 % reported comorbid conditions, and 21% reported patients' behavioural factors in randomized controlled trials. Functioning, environmental factors and inequity-related factors were recorded in 3%, 0% and 9%, respectively, of the systematic reviews; and adherence to interventions, crossovers, and co-interventions in 7%, 0% and 2%, respectively; follow-up percentages in 8%; and adequacy of statistical analyses in 3%.

Conclusion: In all systematic reviews the recording of characteristics of patients, adherence to interventions, follow-up, and statistical analyses in the RCTs was insufficient. The data did not allow assessment of the clinical homogeneity of the randomized controlled trials, or provide justification for meta-analysis, or allow generalizability of the findings.

Keywords: benchmarking method; generalizability; medical journal; risk of bias; systematic review; systematic review and meta-analysis.

Publication types

  • Review

MeSH terms

  • Data Interpretation, Statistical*
  • Humans
  • Meta-Analysis as Topic
  • Periodicals as Topic*
  • Randomized Controlled Trials as Topic*
  • Research Design
  • Systematic Reviews as Topic