Uncertainty Quantification Reveals the Importance of Data Variability and Experimental Design Considerations for in Silico Proarrhythmia Risk Assessment

Abstract

The Comprehensive in vitro Proarrhythmia Assay (CiPA) is a global initiative intended to improve drug proarrhythmia risk assessment using a new paradigm of mechanistic assays. Under the CiPA paradigm, the relative risk of drug-induced Torsade de Pointes (TdP) is assessed using an in silico model of the human ventricular action potential (AP) that integrates in vitro pharmacology data from multiple ion channels. Thus, modeling predictions of cardiac risk liability will depend critically on the variability in pharmacology data, and uncertainty quantification (UQ) must comprise an essential component of the in silico assay. This study explores UQ methods that may be incorporated into the CiPA framework. Recently, we proposed a promising in silico TdP risk metric (qNet), which is derived from AP simulations and allows separation of a set of CiPA training compounds into Low, Intermediate, and High TdP risk categories. The purpose of this study was to use UQ to evaluate the robustness of TdP risk separation by qNet. Uncertainty in the model parameters used to describe drug binding and ionic current block was estimated using the non-parametric bootstrap method and a Bayesian inference approach. Uncertainty was then propagated through AP simulations to quantify uncertainty in qNet for each drug. UQ revealed lower uncertainty and more accurate TdP risk stratification by qNet when simulations were run at concentrations below 5× the maximum therapeutic exposure (C_max). However, when drug effects were extrapolated above 10× C_max, UQ showed that qNet could no longer clearly separate drugs by TdP risk. This was because for most of the pharmacology data, the amount of current block measured was <60%, preventing reliable estimation of IC₅₀-values. The results of this study demonstrate that the accuracy of TdP risk prediction depends both on the intrinsic variability in ion channel pharmacology data as well as on experimental design considerations that preclude an accurate determination of drug IC₅₀-values in vitro. Thus, we demonstrate that UQ provides valuable information about in silico modeling predictions that can inform future proarrhythmic risk evaluation of drugs under the CiPA paradigm.

Keywords: Torsade de Pointes; action potential; cardiac electrophysiology; computational modeling; experimental variability; ion channel; pharmacology; uncertainty quantification.

Figure 1

Uncertainty in bepridil-hERG binding kinetics. (A) The joint probability distribution of K_max (maximum drug effect at saturating concentrations), K_u (rate of drug unbinding), n (Hill coefficient of drug binding), EC₅₀ⁿ (nth power of the half-maximal drug concentration), and V_halftrap (drug trapping potential) was estimated by bootstrapping. Plots on the diagonal show the marginal histograms of each parameter (log-transformed in some cases). Plots below the diagonal show pairwise scatter plots of the fitted parameters for 2,000 bootstrap samples. (B) Kinetics of hERG block during 10 sweeps of a modified Milnes voltage-clamp protocol (Milnes et al., ; Li et al., 2017). Shaded areas show the range of block produced by the parameters from (A). Lines show the experimental results used to fit the data (down-sampled 5× for clarity).

Figure 2

Uncertainty in drug trapping for the 12 CiPA training drugs. Fitted V_halftrap-values (points) are plotted along the curve defining the resulting steady-state fraction of open-bound to close-bound channels (O_bound/C_bound) at V_m = −80 mV. The 95% CIs (horizontal error bars) were estimated with bootstrapping. High TdP-risk drugs are in red, Intermediate-risk drugs are in blue, and Low-risk drugs are in green. Intermediate-risk drugs were indistinguishable from Low- and High-risk drugs.

Figure 3

Uncertainty in the dose-response relationship of late sodium current (I_NaL) block by ranolazine (A,B) and dofetilide (C,D). (A,C) show the joint distribution of pIC₅₀ and Hill coefficient (h)-values, estimated with a Bayesian inference approach. Marginal histograms are displayed on the diagonal plots, and pairwise scatter plots are below the diagonal (2,000 samples per drug). IC₅₀-values are in nM. (B,D) show the dose-response relationships for the two drugs. Solid lines show the Hill equation defined by IC₅₀- and h-values from Li et al. (2017). Shaded areas denote the 95% CI of the percentage block at each concentration, as determined by the parameters in (A,C). Circles are the experimental values used to fit the dose-response curves. Vertical dotted lines indicate the limits of the concentration range used in AP simulations (1−25× C_max).

Figure 4

Uncertainty in dose-response curves at extrapolated drug concentrations. Current block experiments were performed for six ionic currents (see legend) with the 12 CiPA training drugs (72 drug-current combinations total with 19 excluded, see Table 3). Dose-response curves were fitted for each experiment and extrapolated above the highest experimentally tested drug concentration (C_high). Uncertainty in dose-response curves was quantified at 1×, 2×, 3×, and 10× C_high as the width of the 95% CI for the predicted percentage block, plotted as a function of the mean experimentally observed block at 1× C_high. Vertical dotted line is drawn at 60% observed mean block, denoting an approximate lower limit on the mean block that was observed at 1× C_high in experiments for which uncertainty remained low (<16%) at higher concentrations.

Figure 5

Repolarization and depolarization abnormalities in AP simulations. (A) Traces showing representative examples of beats with normal APs (solid), EADs (dashed), or depolarization failure (dotted). (B,C) The percentage of uncertainty-input simulations (2,000 total) in which EADs occurred (B) or which had complete depolarization failure (C) is shown as a function of drug concentration in (B,C), respectively. Only results for drugs that had these events at the simulated concentrations (1−25× C_max) are plotted. (Note that ranolazine had 19 simulations with EADs at 25× C_max; verapamil only had one instance of depolarization failure occurring at 25× C_max.) Markers indicate whether simulations with fixed inputs produced normal Aps (circles), EADs (triangles), or depolarization failure (squares).

Figure 6

Uncertainty in qNet for the 12 CiPA training drugs. Violin plots are shown for qNet distributions at 1× (A) and 10× (B) C_max, based on uncertainty-input simulations. Dotted line indicates the control (no drug) value of qNet. (C) qNet at 1−10× C_max (1× increments) and 15−25× C_max (5× increments). Shaded areas indicate the 95% CIs of qNet obtained from uncertainty-input simulations. Points indicate the highest simulated concentration for which complete experimental data on six non-hERG currents were available. Fixed-input results are shown below (solid lines) or above (dotted lines) this concentration. Likewise, uncertainty-input results are indicated below (dark shaded areas) or above (light shaded areas) this concentration. Simulations with depolarization failure (Figure 5B) were excluded from the results. For all panels, High TdP-risk drugs are in red, Intermediate-risk drugs are in blue, and Low-risk drugs are in green.

Figure 7

Cross validation of TdP risk stratification with uncertainty quantification. LOOCV was performed at each concentration to assess TdP risk stratification performance. Prediction error for each drug was obtained by training on qNet distribution samples from all other drugs and calculating the mean classification error of the test drug's samples. (A) LOOCV at 1−25× C_max. Markers show the prediction errors for each drug when it was “left out,” as indicated in the legend. Black points and error bars are the mean + standard deviation (SD) of prediction errors at each concentration. High TdP-risk drugs are in red, Intermediate-risk drugs are in blue, and Low-risk drugs are in green. (B) LOOCV at 1−4× C_max was repeated with the drug effects for a particular ionic current removed. Black points are the mean prediction errors from (A). Markers show the mean prediction errors that resulted when drug effects on the ionic current indicated in the legend were omitted from simulations.