Purpose: We provide a systematic assessment of the quality and accuracy of statistical reporting in the urology literature.
Materials and methods: All original research publications with adult human subjects in a single issue (August 2004) of 4 leading urology journals were identified for formal review. A standardized evaluation form was developed in consultation with an experienced biostatistician and subsequently tested. Two independent reviewers with at least 1 year of formal training in research design and biostatistics who were blinded to authors and institutions reviewed each article. Discrepancies were settled by consensus and/or adjudication by the biostatistician.
Results: Of the 169 articles screened 97 met eligibility criteria for review. Cohort (43 of 97 or 44%) or cross-sectional (28 of 97 or 29%) designs comprised the majority of these studies. Only 10 randomized clinical trials (12.4%) were identified. Statistical tests were identified in 83 studies (93%). Overall 69 of 83 studies (71%) providing statistical comparisons had at least 1 statistical error, including using the wrong test for the data type in 28%, inappropriate use of a parametric test in 22% and failure to account for multiple comparisons in 65%. In studies applying multivariate analysis (29%) over fitting the model with too many variables was the most common statistical flaw (39%).
Conclusions: This formal review suggests that statistical methods are often used inappropriately in the urology literature, thereby, potentially undermining the validity of study results and conclusions. An effort to raise the awareness of appropriate statistical techniques through postgraduate education appears indicated.