The fragility of findings of randomized controlled trials in shoulder and elbow surgery

Abstract

Background: Considered the gold standard of study designs, randomized controlled trials' (RCTs) results shape clinical practice, effect policy, and influence reimbursement. The fragility index (FI) can be used to quantitate the relative robustness of RCT results, with higher scores indicating more stout results. Unfortunately, most RCTs in surgery have fragile results. The aim of this study was to report on the FI in addition to a qualitative assessment of recent RCTs within the field of shoulder and elbow surgery.

Methods: A systematic review was performed identifying recently published shoulder/elbow RCTs that included 1:1 allocated parallel study arms, dichotomous primary outcome variables, and statistical significance. The FI was calculated by sequentially modifying outcome groups by exchanging a nonevent in one group to an event until the P value for the outcome comparison, as calculated by the Fisher exact test, was increased above the .05 threshold.

Results: Thirty RCTs were included. The median FI was 4. Sixty percent trials had a FI of 2 or less. Fifty-three percent studies reported that participants were lost to follow-up. In 87.5% of these studies, the losses to follow-up exceeded their respective FIs. Only 53% of studies defined a primary outcome variable and 60% studies performed a prestudy power analysis.

Conclusions: The median FI reported in the recent shoulder/elbow literature is 4; however, a high proportion of included RCTs display significant methodological concerns. The FI is a useful adjunct to analyze RCT results, but careful analysis of trial methods should be employed in each circumstance before drawing conclusions.

Keywords: Fragility index; evidence-based medicine; fragility quotient; loss to follow-up; randomized control trial; shoulder and elbow surgery.