Adding decision models to systematic reviews: informing a framework for deciding when and how to do so

Abstract

Objective: Decision models are sometimes used alongside systematic reviews to synthesize evidence. Clarity, however, is lacking about when and how to conduct modeling studies in tandem with systematic reviews, as well as about how to evaluate and present model results. The objective of this study was to collect and analyze information from various sources to inform the development of a framework for deciding when and how a decision model should be added to a systematic review.

Methods: We collected data through 1) review and analysis of evidence reports that used decision models; 2) review and synthesis of current best practices for the development of decision models; 3) interviews of Evidence-Based Practice Center directors and selected staff, United States Preventive Services Task Force members, and decision modelers who developed models used by the United States Preventive Services Task Force; and 4) a focus group of expert modelers.

Results: Models are well suited to address gaps in the literature, better suited for certain types of research questions, and essential for determining the value of information relating to future research. Opinions differ regarding whether model outputs constitute evidence, but interviewees expressed concern over the lack of standards and directions in grading and reporting such "evidence." Interviews of stakeholders and modelers revealed the importance of communication and presentation of model results as well as the importance of model literacy and involvement of stakeholders.

Conclusions: The study demonstrates the need for a framework for deciding when and how to use models alongside systematic reviews and provides information to develop such a framework.