XenoNet: Inference and Likelihood of Intermediate Metabolite Formation

Abstract

Drug metabolism is a common cause of adverse drug reactions. Drug molecules can be metabolized into reactive metabolites, which can conjugate to biomolecules, like protein and DNA, in a process termed bioactivation. To mitigate adverse reactions caused by bioactivation, both experimental and computational screening assays are utilized. Experimental assays for assessing the formation of reactive metabolites are low throughput and expensive to perform, so they are often reserved until later stages of the drug development pipeline when the drug candidate pools are already significantly narrowed. In contrast, computational methods are high throughput and cheap to perform to screen thousands to millions of compounds for potentially toxic molecules during the early stages of the drug development pipeline. Commonly used computational methods focus on detecting and structurally characterizing reactive metabolite-biomolecule adducts or predicting sites on a drug molecule that are liable to form reactive metabolites. However, such methods are often only concerned with the structure of the initial drug molecule or of the adduct formed when a biomolecule conjugates to a reactive metabolite. Thus, these methods are likely to miss intermediate metabolites that may lead to subsequent reactive metabolite formation. To address these shortcomings, we create XenoNet, a metabolic network predictor, that can take a pair of a substrate and a target product as input and (1) enumerate pathways, or sequences of intermediate metabolite structures, between the pair, and (2) compute the likelihood of those pathways and intermediate metabolites. We validate XenoNet on a large, chemically diverse data set of 17 054 metabolic networks built from a literature-derived reaction database. Each metabolic network has a defined substrate molecule that has been experimentally observed to undergo metabolism into a defined product metabolite. XenoNet can predict experimentally observed pathways and intermediate metabolites linking the input substrate and product pair with a recall of 88 and 46%, respectively. Using likelihood scoring, XenoNet also achieves a top-one pathway and intermediate metabolite accuracy of 93.6 and 51.9%, respectively. We further validate XenoNet against prior methods for metabolite prediction. XenoNet significantly outperforms all prior methods across multiple metrics. XenoNet is available at https://swami.wustl.edu/xenonet.

Figure 1.

Metabolic network data set construction and overview of XenoNet. (A) Multiple experimentally observed reactions from the AMD can be linked to an annotated network. Using these annotated networks, we can evaluate how well different metabolic network-generating algorithms can infer observed intermediate metabolites. (B) XenoNet is a metabolic network predictor that, given a substrate and a target product as inputs, can infer the metabolic pathways connecting two input molecules and the corresponding likelihood of each pathway. In XenoNet, the Metabolic Forest algorithm is applied iteratively to generate a tree of potential pathways that span multiple metabolic transformations. During construction of this tree, pathways between the starting molecule and a target molecule can be enumerated. The likelihood of each step in a pathway can then be computed from the five-color predictions by Rainbow XenoSite. The five colors are used to allow for ease of use when visualizing the networks that XenoNet generates. Each of the major groups corresponding to one of the five colors is further subdivided into more nuanced reaction-type classes that are used to annotate the edges in the generated network. These more detailed edge annotations are accessible through the XenoNet network object. XenoNet’s predicted metabolic pathways are stored using a graph-based data structure where each molecule is a node and each metabolic transformation is an edge.

Figure 2.

XenoNet can infer the metabolic pathways connecting a substrate and a product from their structures in three steps. First, using Metabolic Forest, a depth-first search scans across a space of possible metabolite structures for paths that terminate at the given product metabolite. Second, the discovered paths are used to construct a new metabolic network with the input substrate and product as terminal nodes. Third, Rainbow XenoSite yields predictions on the metabolic transformation edges of the constructed metabolic network. As an example, we show here how the process works when the model is given 1,3-butadiene (in solid-line box) and hydroxymethylvinyl ketone (in dash-line box) as its substrate and product pair inputs. The search starts at 1,3-butadiene and explores as far as possible along a branch of metabolic transformations up to a limited depth, before backtracking to continue the search along other branches (left). Only pathways that go from 1,3-butadiene to hydroxymethylvinyl ketone are retained to construct a new metabolic network (right). Once all metabolites and directed edges linking them are in the network, the network is passed into Rainbow XenoSite to compute the likelihood for each metabolic transformation, shown as the numbers next to the edges.

Figure 3.

Top-N and substructure matching heuristics. To limit the branching factor of potential metabolic trees, we use top-N and substructure matching heuristics. As an example, let us consider a pair of a substrate (in solid-line box) and a target product (in dash-line box), as shown above. For the first generation of metabolites, XenoNet discovers seven structures, one of which is our target product. However, only the target product and structures that meet top-N and/or substructure heuristics would be considered in the next step of the search and the remaining metabolites would be ignored. The top-N heuristic filters for metabolite structures that receive N highest scores. The numbers next to the arrows are scores assigned by Rainbow XenoSite for the corresponding metabolic transformation. Only two metabolites with the scores of 0.60 and 0.45 pass the Top-2 filter. The substructure matching heuristic filters for metabolite structures produced via transformation at sites expected to lead to the target product. In the substrate structure, the site of the structural difference between the substrate and target compound, i.e., the site of metabolism, is circled in red. In each metabolite structure, the site of structural difference between the metabolite and the substrate is circled in red if it contains the site of metabolism and gray if it does not. Only two metabolites with red circles pass the substructure matching filter. Different combinations of top-N and substructure matching heuristics are used to construct six XenoNet variants.

Figure 4.

Metabolite score. To rank the relative importance of metabolites in a given network, we calculate metabolite scores. There are three steps in this process. (a) First, each metabolic transformation in the network is assigned with a raw prediction by Rainbow XenoSite. (b) Second, the raw prediction is normalized using eq 1. (c) Third, the substrate is assigned a score of 1.0, and downstream metabolites are assigned with scores computed using eq 2. Because a node can only be scored after all of its parents have been scored, the third step is carried out in multiple layers. The dashed-line dividing the network into segments corresponds to the computation of the metabolite scores for subsequent layers over the network. On the first layer, for example, scores could be computed for A. The computed score for A, displayed as numbers adjacent to each metabolite, is 0.25. The number of layers required to compute all metabolite scores is equal to the path of maximum length. The maximum length path in this network requires five steps. Computing metabolite scores for all metabolites in the network shown required five layers.

Figure 5.

Time cost varies greatly between the seven XenoNet variants. In this comparison, each method was allotted 30 min for network generation of each substrate–product pair at a depth limit of three steps from the substrate. (Top) Across 1024 substrate–product pairs, only the substructure matching, top-N specific, and top-N agnostic variants can produce above 70% fully generated networks within the 30 min allotted time. In the same allotted time limit, the optimal thresholds heuristic fully generated almost 50% of the networks. In contrast, while the naive and the combination variants are the least restricted in terms of the metabolite structures generated, they are only able to produce at most 20% fully generated networks within 30 min. (Bottom) A similar trend is found when comparing the run time distributions. The substructure matching, top-N specific, and top-N agnostic variants, on average, take less than 10 min to generate a network. The time distribution for the naive and the combination variants is much broader, and many runs hit the 30 min timeout before producing a fully generated network.

Figure 6.

Path recall calculation. Path recall is a metric designed to measure how well a model captures annotated paths. Here, we show how path recall is calculated for a hypothetical data set of five substrate–product pairs. For each substrate–product pair, a predicted graph is generated within a depth limit of 3. An annotated path from the substrate to the target product is considered as being captured if (1) all of its nodes are contained in a predicted path and (2) the order of traversal through these nodes is the same in the annotated and predicted paths. For each pair of substrate and product molecules, the proportions of annotated paths of a certain length that were captured are calculated. For each length classification, the path recall of a test set is the average of the captured proportions at that specific path length across the entire test set.

Figure 7.

Path recall performance is dependent on the path length. While heuristic approaches capture shorter-length annotated paths better, the naive model is superior in capturing longer-length annotated paths. The performance trend highlights the trade-off between employing heuristics to speed up network generation at the cost of constraining the set of possible child metabolites. The constraining factor grows exponentially as the path length grows.

Figure 8.

Intermediate metabolite recall calculation. The intermediate recall is a metric designed to measure how well the model can infer experimentally observed intermediates, compounds on the paths from the substrate to the target product. Here, we show how the intermediate recall is calculated for a hypothetical data set of five substrate–product pairs. For each substrate–product pair, a predicted graph is generated within a depth limit of 3. For each input substrate–product pair, the proportions of experimentally observed intermediates that can be inferred in the predicted graph of certain minimal depth are calculated. For each minimal depth classification, an intermediate recall of a test set is the average of these depth limit-specific proportions across the entire data set.

Figure 9.

Combined substructure matching and top-N specific heuristic model has the highest intermediate metabolite recall across all depths.

Figure 10.

Recovery rate calculation. Here, we show how the recovery rate is calculated for a set of three hypothetical metabolic networks. (Left) Predicted metabolic networks with a depth limit of one of three different substrates, S₁, S₂, and S₃, and their known metabolites are shown. Metabolites M that are both predicted and experimentally observed are in darker gray. Metabolites M that are predicted but not experimentally observed or vice versa are in lighter gray. (Right) The proportion of networks that have fractions of known metabolites predicted by the model above a certain threshold is computed. In our hypothetical database of three metabolic networks, 100% of substrates have at least 0–20% of their known metabolites predicted, and 66.7% of the substrates (S₁ and S₂) have at least 60% of their known metabolites predicted. Only 33.3% of the substrates (S₁) have at least 70% of their known metabolites predicted.

Figure 11.

XenoNet accurately predicts metabolic networks. Performance of XenoNet across the full metabolic network data set on the same metrics assessed during the comparison between all heuristics on the subset of 1024 networks. (A) Path recall stratified across paths of length 1, 2, and 3. (B) Intermediate metabolite recall stratified across minimum network depth limit required to reach the intermediate metabolite of 2, 3, and 4 or more. (C) Intermediate metabolite recovery across recovery rate thresholds of 0.1 to 1. The results for all three plots are close to the initial results for each metric when validated on the subset of 1024 samples and supports that the comparison between the XenoNet variants generalizes to performance on the full data set.

Figure 12.

Path ranking calculation. This metric is designed to measure how well the predicted path with the highest likelihood of traversal corresponds to a known path in the annotated graph. Here, we show how path ranking is calculated for a hypothetical data set of five substrate–product pairs. The likelihood of each predicted pathway is the logarithmic sum over the scores that Rainbow XenoSite assigns to the metabolic transformations in that pathway. We then rank all predicted pathways in a given network by their likelihoods. If the highest-likelihood path of a given network has an exact match in the annotated path set, then that network is assigned a score of 1. Otherwise, that network is assigned a score of 0.

Figure 13.

XenoNet is superior to published metabolite prediction models. The receiver-operating characteristic curves (ROC) and their corresponding areas under the curve (AUC) are reliable metrics to compare between the models. The AUC of XenoNet, GLORY MaxCoverage, and SyGMa are 73.3, 67.6, and 50.1%, respectively. We were unable to construct a ROC curve and calculate the AUC for BioTransformer based on their publication, but their recall of 0.72 is lower than other models if all of them were set to have the same precision of 0.17.

Figure 14.

XenoNet is superior to GLORY MaxCoverage in terms of the recovery rate. Across all thresholds, XenoNet’s recovery rate is higher than GLORY’s.