EPC Methods: AHRQ End-User Perspectives of Rapid Reviews [Internet]

Excerpt

Objectives: The goal of this project was to understand end-user perspectives on three types of rapid review products: evidence inventories, rapid responses, and rapid reviews. This taxonomy of rapid products was developed in previous work conducted through the Agency for Healthcare Research and Quality's (AHRQ) Evidence-based Practice Center (EPC) Program. We sought to (1) identify critical elements in an evidence synthesis that end-users value, (2) determine impressions of rapid products, and (3) determine where/when/how end-users might use rapid review products and whether this varies by the nature of the decision being made. To ensure the findings were most relevant to AHRQ's EPC Program, we focused on decisionmakers who frequently used reviews from the AHRQ EPC Program.

Methods: Qualitative interviews were conducted with individuals (i.e., Key Informants, KIs) from U.S. organizations representing: guideline developers (n=3), health care provider organizations (n=3), research funders (n=1), and payers/health insurers (n=1). All KIs were familiar with or had used EPC reports (i.e., standard systematic reviews); some also produced (n=3) or had experience using (n=2) rapid products. We elicited perspectives on important characteristics of systematic reviews, users' perspectives on methods employed to streamline reviews, and uses of rapid review products. Two research assistants analyzed content of the transcripts, and two investigators independently reviewed all transcripts and verified themes and subthemes. All themes and subthemes were discussed with the study team.

Results: KIs identified the following as critical for a systematic review: (1) the review was from a reliable source (i.e., conducted by experienced reviewers from an established research organization); (2) the review addressed clinically relevant questions; and (3) there was a trusted relationship between the user and producer. KIs expressed strong preference for the following review methods and characteristics: strength of evidence assessments, quality rating of studies, use of evidence tables, and use of summary tables of results and conclusions. The most acceptable trade-offs to increase reviewer efficiencies were in limiting the literature search and performing single screening of abstracts and full texts for relevance. KIs reported a variety of potential uses for rapid products. In general, KIs perceived rapid products (particularly evidence inventories and rapid responses) as useful interim products to inform downstream investigation (e.g., whether to proceed with a full review or a guideline, direction for future research). Most KIs indicated that analysis/synthesis and quality/strength of evidence was important for decisionmaking. Most KIs could see a use for a rapid review, in particular for guideline development focused on narrow topics, policy decisions when a quick turn-around is needed, decisionmaking for practicing clinicians in nuanced clinical settings, and coverage decisions. Rapid responses and rapid reviews may be more relevant within specific clinical settings or health systems. Conversely, broad/national guidelines often need a traditional systematic review.

Conclusions: This work provides insight into the perspectives of AHRQ end-users on rapid review products, highlighting as important: the credibility of the review producer, relevance of key questions, and close working relationship between the end-user and producer. This work also identified review characteristics and methods that are considered essential for decisionmaking, acceptable methodological tradeoffs, and potential uses of rapid products.