Objective: To assess the methodology of meta-analyses published in leading general and specialist medical journals over a 10-year period.
Study design and setting: Volumes 1993-2002 of four general medicine journals and four specialist journals were searched by hand for meta-analyses including at least five controlled trials. Characteristics were assessed using a standardized questionnaire.
Results: A total of 272 meta-analyses, which included a median of 11 trials (range 5-195), were assessed. Most (81%) were published in general medicine journals. The median (range) number of databases searched increased from 1 (1-9) in 1993/1994 to 3.5 (1-21) in 2001/2002, P<0.0001. The proportion of meta-analyses including searches by hand (10% in 1993/1994, 25% in 2001/2002, P=0.005), searches of the grey literature (29%, 51%, P=0.010 by chi-square test), and of trial registers (10%, 32%, P=0.025) also increased. Assessments of the quality of trials also became more common (45%, 70%, P=0.008), including whether allocation of patients to treatment groups had been concealed (24%, 60%, P=0.001). The methodological and reporting quality was consistently higher in general medicine compared to specialist journals.
Conclusion: Many meta-analyses published in leading journals have important methodological limitations. The situation has improved in recent years but considerable room for further improvements remains.