Systematic reviews synthesized evidence without consistent quality assessment of primary studies examining epidemiology of chronic diseases

Abstract

Objective: To evaluate how systematic reviews assess the quality of primary studies of incidence, prevalence, or risk factors for chronic diseases.

Study design and setting: We searched several databases, identified 145 systematic reviews, and evaluated methods of quality assessment and quantitative synthesis of evidence by external or internal validity or overall quality of primary studies.

Results: Of 145 reviews, 54 (37%) reported a planned quality assessment of primary studies with checklists or scales and 26 (18%) reported evaluation of some selected quality criteria. Thirty-nine percent of reviews judged appropriateness of sampling and proper controls for confounding factors in primary studies. Twelve percent synthesized evidence by overall quality, 17% by design, 42% by criteria of internal validity, and 24% by external validity of primary studies. Masking of quality assessment was conducted on 2.1% of reviews and 4% tested interobserver agreement for quality assessment.

Conclusion: Evaluation of internal and external validity of primary studies is uncommon in systematic reviews of studies of incidence, prevalence, or risk factors for chronic diseases. Inconsistent quality assessment practices reflect the absence of uniformly accepted standards and tools to examine the quality of observational nontherapeutic studies.