Development and validation of a simple web-based tool for early prediction of COVID-19-associated death in kidney transplant recipients

Abstract

This analysis, using data from the Brazilian kidney transplant (KT) COVID-19 study, seeks to develop a prediction score to assist in COVID-19 risk stratification in KT recipients. In this study, 1379 patients (35 sites) were enrolled, and a machine learning approach was used to fit models in a derivation cohort. A reduced Elastic Net model was selected, and the accuracy to predict the 28-day fatality after the COVID-19 diagnosis, assessed by the area under the ROC curve (AUC-ROC), was confirmed in a validation cohort. The better calibration values were used to build the applicable ImAgeS score. The 28-day fatality rate was 17% (n = 235), which was associated with increasing age, hypertension and cardiovascular disease, higher body mass index, dyspnea, and use of mycophenolate acid or azathioprine. Higher kidney graft function, longer time of symptoms until COVID-19 diagnosis, presence of anosmia or coryza, and use of mTOR inhibitor were associated with reduced risk of death. The coefficients of the best model were used to build the predictive score, which achieved an AUC-ROC of 0.767 (95% CI 0.698-0.834) in the validation cohort. In conclusion, the easily applicable predictive model could assist health care practitioners in identifying non-hospitalized kidney transplant patients that may require more intensive monitoring. Trial registration: ClinicalTrials.gov NCT04494776.

Keywords: clinical research/practice; complication: infectious; health services and outcomes research; infection and infectious agents - viral; infectious disease; kidney transplantation/nephrology.

FIGURE 1

Participant flow diagram and proportion of patients enrolled in the derivation and validation cohorts. Using a random split, 1,035 patients were grouped in the training cohort (training data set), which represents 75% of the entire cohort, whereas 344 patients were grouped in the validation cohort (test data set). A more detailed flow diagram of the population can be consulted in the supplementary material (Figure S1)

FIGURE 2

Calibration plot of COVID-19 mortality models in the validation cohort: (A) XGBoost full model, (B) XGBoost reduced model, (C) Elastic Net full model, (D) Elastic Net reduced model. Gray line represents perfectly calibrated model, solid black line represents optimism corrected model using logistic calibration, and doted black line represents optimism corrected model using nonparametric calibration

FIGURE 3

AUC-ROC in the derivation cohort of COVID-19-associated death. The red line represents the ROC curve of the reduced Elastic Net, which achieved the best performance to predict 28-day mortality in the derivation cohort: 0.767 (95% CI 0.698–0.834) [Color figure can be viewed at wileyonlinelibrary.com]

FIGURE 4

Confusion matrix of 28-day COVID-19-associated death in the derivation cohort. The lower number of patients for whom the model did not predict the outcome but it occurred in the real life was achieved by the reduced Elastic net (n = 15) [Color figure can be viewed at wileyonlinelibrary.com]

FIGURE 5

Coefficients of Elastic Net of COVID-19-associated death model. The plot represents the variable importance. The red bars represent the variables related to the probability of death, whereas the blue bars were related to the probability of surviving. The model was fitted with 15 predictors and natural splines in the variables age and eGFR were derived. The natural splines computed a different risk for each stratum aiming to capture the non-linear association between these predictors and outcome. AZA, azathioprine; BMI, body mass index; ESKD, end stage kidney disease; DM, diabetes mellitus; eGFR, estimated glomerular filtration rate; MPAA, mycophenolate acid analogs; mTOR, mammalian target of rapamycin [Color figure can be viewed at wileyonlinelibrary.com]

FIGURE 6

Shapley Additive Explanations (SHAP plot) showing the contribution of each predictor in COVID-19-associated death score in simulated transplant patients. The red bars represent variables with a positive coefficient that means a positive association between the predictor and the outcome, while the blue bars represent variables with a negative coefficient that means an inverse association between the predictor and the outcome. eGFR, estimated glomerular filtration rate; mTOR, mammalian target of rapamycin [Color figure can be viewed at wileyonlinelibrary.com]