High-dimensional multiple imputation for partially observed confounders including natural language processing-derived auxiliary covariates

Abstract

Multiple imputation (MI) models can be improved with auxiliary covariates (ACs), but their performance in high-dimensional data remains unclear. We aimed to develop and compare high-dimensional MI (HDMI) methods using structured and natural language processing (NLP)-derived AC in studies with partially observed confounders. We conducted a plasmode simulation with acute kidney injury as outcome and simulated 100 cohorts with a null treatment effect, incorporating creatinine labs, atrial fibrillation (AFib), and other investigator-derived confounders in the outcome generation. Missingness was imposed on creatinine based on creatinine itself and AFib. Different HDMI candidate ACs were created using structured and NLP-derived features, and we mimicked scenarios where AFib was unobserved by omitting it from all analyses. Using the least absolute shrinkage and selection operator, we selected HDMI covariates for MI and propensity score models. The treatment effect was estimated after propensity score matching in MI datasets, and HDMI methods were compared to baseline imputation and complete case analysis. High-dimensional MI using claims data showed the lowest bias (0.072). Combining claims and sentence embeddings led to an improvement in the efficiency with a root mean square error (RMSE) of 0.173 and 94% coverage. Natural language processing-derived AC alone did not outperform baseline MI. High-dimensional MI approaches may decrease bias in studies where confounder missingness depends on unobserved factors.

Keywords: EHR; NLP; confounding; missing data; real-world evidence.