Analysis of quality of clinical practice guidelines for otorhinolaryngology in China

Abstract

Objective: To evaluate the quality of clinical practice guidelines (CPGs) for otorhinolaryngology in China.

Materials and methods: A systematic search of relevant literature databases (CBM, WANFANG, VIP, CNKI, China Guideline Clearinghouse) published between 1978 and March 2012 was undertaken to identify and select CPGs related to otorhinolaryngology. Four independent reviewers assessed the eligible guidelines using the Appraisal of Guidelines for Research and Evaluation (AGREE II) instrument. Their degree of agreement was evaluated using the intraclass correlation coefficient (ICC).

Result: From 170 citations, 21 relevant guidelines were included. The overall agreement among reviewers was moderate (ICC = 0.87; 95% confidence interval [CI], 0.78-0.91). The scores for each of the AGREE domains were the following: "scope and purpose" (mean ± standard error [SE] = 45.4±4.4; ICC = 0.92), "stakeholder involvement" (mean ± SE = 30.4±3.1; ICC = 0.81), "rigor of development" (mean ± SE = 20.9±2.8; ICC = 0.87), "clarity of presentation" (mean ± SE = 48.8±3.7; ICC = 0.80), "applicability" (mean ± SE = 12.6±1.7; ICC = 0.72), and "editorial independence" (mean ± SE = 6.2±0.8; ICC = 0.76). Three guidelines (14%) mentioned updates, and the average update frequency was 7 years. None used the GRADE system.

Conclusion: The quality of otorhinolaryngology guidelines in China is low. Greater efforts are needed to provide high-quality guidelines that serve as a useful and reliable tool for clinical decision-making in this field.

Figure 1. Flowchart of systematic search.

Figure 2. Comparison of the quality of the guidelines studied in this report with those studied by Chen (Chinese CPG) and Alonso-Coello (world CPG).

This graph depicts AGREE scores: (A) Mean domain scores as reported in this study. (B) Mean domain scores evaluated on clinical guidelines in China by ChenYL. (C) Mean domain scores as reported by Pablo Alonso-Coello.