The Yale Guideline Recommendation Corpus: a representative sample of the knowledge content of guidelines

Abstract

Objective: To develop and characterize a large, representative sample of guideline recommendations that can be used to better understand how current recommendations are written and to test the adequacy of guideline models. We refer to this sample as the Yale Guideline Recommendation Corpus (YGRC).

Method: To develop the YGRC, we extracted recommendations from guidelines downloaded from the National Guideline Clearinghouse (NGC). We evaluated the representativeness of the YGRC by comparing the frequency of use of controlled vocabulary terms in the YGRC sample and in the NGC. We examined semantic and formatting indicators that were used to denote recommendation statements.

Results: In the course of reviewing 7527 recommendation statements, we extracted 1275 recommendations from the NGC and characterized the guidelines from which they were derived. Both semantic and formatting indicators were used inconsistently to denote recommendations. Recommendation statements were not reliably identifiable in 31.6% (310/982) of the guidelines and many recommendations were not executable as written. We also found variability and inconsistency in the way strength of recommendation is currently reported. Over half of the recommendations (52.7%), did not indicate strength, while 6.5% inaccurately indicated strength.

Conclusion: The YGRC provides a representative sample of current guideline recommendations and demonstrates considerable variability and inconsistency in the way recommendations are written and in the way the recommendation strength is currently reported.