Two methods provide similar signals for the need to update systematic reviews

Abstract

Objective: Apply and compare two methods that identify signals for the need to update systematic reviews, using three Evidence-based Practice Center reports on omega-3 fatty acids as test cases.

Study design and setting: We applied the RAND method, which uses domain (subject matter) expert guidance, and a modified Ottawa method, which uses quantitative and qualitative signals. For both methods, we conducted focused electronic literature searches of recent studies using the key terms from the original reports. We assessed the agreement between the methods and qualitatively assessed the merits of each system.

Results: Agreement between the two methods was "substantial" or better (kappa>0.62) in three of the four systematic reviews. Overall agreement between the methods was "substantial" (kappa=0.64, 95% confidence interval [CI] 0.45-0.83).

Conclusion: The RAND and modified Ottawa methods appear to provide similar signals for the possible need to update systematic reviews in this pilot study. Future evaluation with a broader range of clinical topics and eventual comparisons between signals to update reports and the results of full evidence review updates will be needed. We propose a hybrid approach combining the best features of both methods, which should allow efficient review and assessment of the need to update.

Fig. 1

Search results and article flow for each of the three omega-3 fatty acid evidence reports.