Background: Statistical analysis of systematic reviews allows the results of previous studies to be combined and synthesized to assess the overall health effect of the intervention in question. Systematic reviews can also be used to guide the creation of clinical practice guidelines and are considered to have a high level of evidence. Thus, it is important that their methodological quality is of the highest standard. Publication bias presents 2 problems: (1) studies with significant results may be overrepresented in systematic reviews and meta-analyses ("false positives") and (2) studies without significant results may not be included in systematic reviews and meta-analyses ("false negatives") because each study, on its own, was underpowered, meaning that some treatment options that may have clinical benefit will not be adopted.
Methods: We performed a study to evaluate the techniques used by authors to report and evaluate publication bias in the top 10 orthopaedic journals as well as 3 orthopaedic-related Cochrane groups. Two authors independently screened the titles and abstracts to identify systematic reviews and meta-analyses. We assessed publication bias in the systematic reviews that did not assess publication bias themselves.
Results: Our final sample included 694 systematic reviews or meta-analyses that met our inclusion criteria. Our review included 502 studies (72%) that focused on clinical outcomes, with the majority of the remaining studies focused on predictive and prognostic accuracy (20%) or diagnostic accuracy (5%). Publication bias was discussed in 295 (42.5%) of the included studies and was assessed in 135 (19.5%). Of the studies that assessed publication bias, 31.9% demonstrated evidence of publication bias. Only 43% and 22% of studies that involved use of the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines discussed and assessed publication bias, respectively.
Conclusions: Publication bias is infrequently discussed and assessed in the high-impact orthopaedic literature. Furthermore, nearly one-third of the studies that assessed for publication bias demonstrated evidence of publication bias. In addition to these shortcomings, fewer than half of these studies involved use of the PRISMA guidelines and yet only one-fourth of the studies assessed for publication bias.
Clinical relevance: By understanding the degree to which publication bias is discussed and presented in high-impact orthopaedic literature, changes can be made by journals and researchers alike to improve the overall quality of research produced and reported.