GRADE guidelines 17: assessing the risk of bias associated with missing participant outcome data in a body of evidence

Gordon H Guyatt; Shanil Ebrahim; Pablo Alonso-Coello; Bradley C Johnston; Alexander G Mathioudakis; Matthias Briel; Reem A Mustafa; Xin Sun; Stephen D Walter; Diane Heels-Ansdell; Ignacio Neumann; Lara A Kahale; Alfonso Iorio; Joerg Meerpohl; Holger J Schünemann; Elie A Akl

doi:10.1016/j.jclinepi.2017.05.005

GRADE guidelines 17: assessing the risk of bias associated with missing participant outcome data in a body of evidence

J Clin Epidemiol. 2017 Jul:87:14-22. doi: 10.1016/j.jclinepi.2017.05.005. Epub 2017 May 18.

Authors

Gordon H Guyatt¹, Shanil Ebrahim², Pablo Alonso-Coello³, Bradley C Johnston⁴, Alexander G Mathioudakis⁵, Matthias Briel⁶, Reem A Mustafa⁷, Xin Sun⁸, Stephen D Walter⁹, Diane Heels-Ansdell⁹, Ignacio Neumann¹⁰, Lara A Kahale¹¹, Alfonso Iorio¹, Joerg Meerpohl¹², Holger J Schünemann¹, Elie A Akl¹³

Affiliations

¹ Department of Health Research Methods, Evidence and Impact, McMaster University, 1200 Main St. West, Hamilton L8S 4K1, Canada; Department of Medicine, McMaster University, 1200 Main St. West, Hamilton L8S 4K1, Canada.
² Department of Health Research Methods, Evidence and Impact, McMaster University, 1200 Main St. West, Hamilton L8S 4K1, Canada; Systematic Overviews through Advancing Research Technology (SORT), Child Health Evaluative Sciences, The Hospital for Sick Children Research Institute, 555 University Ave, Toronto, ON M5G 1X8, Canada.
³ Department of Health Research Methods, Evidence and Impact, McMaster University, 1200 Main St. West, Hamilton L8S 4K1, Canada; Iberoamerican Cochrane Centre, CIBERESP-IIB Sant Pau, Casa de Convalescéncia, 4 th floor, C. Sant Antoni Maria Claret 171, Barcelona 08041, Spain.
⁴ Department of Health Research Methods, Evidence and Impact, McMaster University, 1200 Main St. West, Hamilton L8S 4K1, Canada; Systematic Overviews through Advancing Research Technology (SORT), Child Health Evaluative Sciences, The Hospital for Sick Children Research Institute, 555 University Ave, Toronto, ON M5G 1X8, Canada; Department of Anesthesia and Pain Medicine, The Hospital for Sick Children, University of Toronto, 555 University Ave, Toronto, ON M5G 1X8, Canada; Institute for Health Policy, Management and Evaluation, Dalla Lana School of Public Health, University of Toronto, 155 College St, Toronto, ON M5T 3M7, Canada.
⁵ Iberoamerican Cochrane Centre, CIBERESP-IIB Sant Pau, Casa de Convalescéncia, 4 th floor, C. Sant Antoni Maria Claret 171, Barcelona 08041, Spain.
⁶ Department of Health Research Methods, Evidence and Impact, McMaster University, 1200 Main St. West, Hamilton L8S 4K1, Canada; Department of Clinical Research, Basel Institute for Clinical Epidemiology and Biostatistics, University Hospital Basel, Hebelstrasse 10, Basel 4056, Switzerland.
⁷ Department of Health Research Methods, Evidence and Impact, McMaster University, 1200 Main St. West, Hamilton L8S 4K1, Canada; Department of Internal Medicine, Kansas University Medical Center, 3901 Rainbow Blvd, Kansas City, KS MS3002, USA.
⁸ Chinese Evidence-based Medicine Center, West China Hospital, Sichuan University, Chengdu 610041, China.
⁹ Department of Health Research Methods, Evidence and Impact, McMaster University, 1200 Main St. West, Hamilton L8S 4K1, Canada.
¹⁰ Department of Internal Medicine, Pontificia Universidad Catolica de Chile, Av Libertador Bernardo O'Higgins 340, Santiago, Región Metropolitana, Chile.
¹¹ Department of Internal Medicine, American University of Beirut, Riad-El-Solh Beirut, Beirut 1107 2020, Lebanon.
¹² Cochrane Germany, Medical Center-University of Freiburg, Breisacher Strasse 153, Freiburg 79110, Germany; Centre de Recherche Épidémiologie et Statistique Sorbonne Paris Cité-U1153, Inserm/Université Paris Descartes, Cochrane France, Hôpital Hôtel-Dieu, 1 place du Parvis Notre Dame, Paris Cedex 04 75181, France.
¹³ Department of Health Research Methods, Evidence and Impact, McMaster University, 1200 Main St. West, Hamilton L8S 4K1, Canada; Department of Internal Medicine, American University of Beirut, Riad-El-Solh Beirut, Beirut 1107 2020, Lebanon. Electronic address: ea32@aub.edu.lb.

PMID: 28529188
DOI: 10.1016/j.jclinepi.2017.05.005

Abstract

Objective: To provide GRADE guidance for assessing risk of bias across an entire body of evidence consequent on missing data for systematic reviews of both binary and continuous outcomes.

Study design and setting: Systematic survey of published methodological research, iterative discussions, testing in systematic reviews, and feedback from the GRADE Working Group.

Results: Approaches begin with a primary meta-analysis using a complete case analysis followed by sensitivity meta-analyses imputing, in each study, data for those with missing data, and then pooling across studies. For binary outcomes, we suggest use of "plausible worst case" in which review authors assume that those with missing data in treatment arms have proportionally higher event rates than those followed successfully. For continuous outcomes, imputed mean values come from other studies within the systematic review and the standard deviation (SD) from the median SDs of the control arms of all studies.

Conclusions: If the results of the primary meta-analysis are robust to the most extreme assumptions viewed as plausible, one does not rate down certainty in the evidence for risk of bias due to missing participant outcome data. If the results prove not robust to plausible assumptions, one would rate down certainty in the evidence for risk of bias.

Keywords: GRADE; Missing participant data; Risk of bias; Systematic reviews; Trials.

Publication types

Practice Guideline

MeSH terms

Bias
Humans
Lost to Follow-Up
Patient Dropouts / statistics & numerical data*
Research Design*
Review Literature as Topic*
Risk Assessment