A systematic review of tools used to assess the quality of observational studies that examine incidence or prevalence and risk factors for diseases

J Clin Epidemiol. 2010 Oct;63(10):1061-70. doi: 10.1016/j.jclinepi.2010.04.014.


Objective: To create a comprehensive evaluation of checklists and scales used to evaluate observational studies that examine incidence or prevalence and risk factors for diseases.

Study design: We did a literature search of several databases to abstract format, content, development, and validation of the tools.

Results: We identified 46 scales and 51 checklists. Forty-seven of these tools were created for therapeutic studies, 48 for risk factors, and 5 for incidence studies. Forty-seven percent were modifications of previously published peer-reviewed appraisals, 18% were developed based on methodological standards, and 35% did not report development. Twenty-two percent reported reliability and 10% the validation procedure. Tools did not discriminate poor reporting vs. methodological quality of studies or external vs. internal validity; 35% categorize quality by the presence of predefined major flaws in design or by total score from the scale. Level of evidence was proposed in 22% of the tools by criteria of causality or internal validity of the studies. Evaluation required different degrees of subjectivity.

Conclusions: Format, length, and content varied substantially across available checklists and scales. Development, validation, and reliability were not consistently reported. Transparent objective quality assessments should be developed in the future.

Publication types

  • Review
  • Systematic Review

MeSH terms

  • Bias
  • Checklist / methods*
  • Disease / etiology*
  • Epidemiology*
  • Humans
  • Incidence
  • Prevalence
  • Quality Control
  • Reproducibility of Results
  • Research Design / standards*
  • Risk Factors