Insufficient information size and potentially false results in orthodontic meta-analyses using trial sequential analysis

Abstract

Background: Meta-analysis (MA) is a common mathematical method used in systematic reviews (SRs) to gather data from different studies. MA may result in overestimation or underestimation of the effect due to systematic and random errors. Trial sequential analysis (TSA) has been used to overcome the limitations of conventional MA. The objective of this study was to re-evaluate the findings of orthodontic MAs using TSA to investigate the conclusiveness of the effect and to estimate the required information size (IS).

Methods: Orthodontic SRs with MA published between 1 January 2013 and 31 December 2022 in the leading orthodontic journals were sourced. Data from arm-level MAs were extracted and re-entered in TSA software.

Results: A total of 180 papers were assessed against the inclusion criteria. Finally, 50 SRs with MAs were included. TSA found that almost half of the MAs with significant results demonstrated firm evidence of effect. While 38% of the significant MAs confirmed potentially spurious evidence of effect. In contrast, only one MA with insignificant findings showed a lack of effect. Furthermore, a larger number of patients were needed when the evidence of the effect was absent.

Conclusions: TSA revealed that many orthodontic MAs have potentially false-positive results and have insufficient IS.

Keywords: TSA; meta-analysis; orthodontic; systematic reviews; trial sequential analysis.