GRADE Notes 2: Criteria for searching non-randomized or indirect evidence should be defined early in the guideline production process

Abstract

Objective: To discuss two alternative approaches for complementing the body of direct evidence from Randomized Controlled Trials (RCTs) when it is judged insufficient from a guideline panel making recommendations. The approaches included expanding the evidence's body to non-randomises studies on the population of interest or to RCTs on indirect populations.

Study design and setting: In this report, we adopt the perspective of an evidence review team developing guidelines following the GRADE approach. Our experience is based on the development of two evidence-based guidelines promoted by The Italian National Institute of Health (ISS) and focusing on diagnosis and treatment of Autism Spectrum Disorders (ASD) in children/adolescents and adults.

Results: We left panel members deciding case by case whether the direct evidence from RCTs was sufficient or not and indicating which alternative to implement. This strategy presented unanticipated challenges both from an organizational and methodological standpoint.

Conclusion: We suggest an early-stage production of a research protocol to define the criteria for expanding the body of evidence. These criteria should be informed by considerations around the certainty in the evidence, the clinical applicability of the results, feasibility and conflict of interest.

Keywords: Certainty of evidence; Clinical practice guidelines; Grade; Indirectness; Randomized Controlled Trials; Risk of bias.