dendsort: modular leaf ordering methods for dendrogram representations in R

Abstract

Dendrograms are graphical representations of binary tree structures resulting from agglomerative hierarchical clustering. In Life Science, a cluster heat map is a widely accepted visualization technique that utilizes the leaf order of a dendrogram to reorder the rows and columns of the data table. The derived linear order is more meaningful than a random order, because it groups similar items together. However, two consecutive items can be quite dissimilar despite proximity in the order. In addition, there are 2 (n-1) possible orderings given n input elements as the orientation of clusters at each merge can be flipped without affecting the hierarchical structure. We present two modular leaf ordering methods to encode both the monotonic order in which clusters are merged and the nested cluster relationships more faithfully in the resulting dendrogram structure. We compare dendrogram and cluster heat map visualizations created using our heuristics to the default heuristic in R and seriation-based leaf ordering methods. We find that our methods lead to a dendrogram structure with global patterns that are easier to interpret, more legible given a limited display space, and more insightful for some cases. The implementation of methods is available as an R package, named "dendsort", from the CRAN package repository. Further examples, documentations, and the source code are available at [https://bitbucket.org/biovizleuven/dendsort/].

Figure 1.. Cluster heat map of the data matrix from the integrated pathway analysis of gastric cancer from the Cancer Genome Atlas (TCGA) study.

Figure 2.. Hierarchical clustering of a simulated two-dimensional data set.

( A) A scatterplot of the ten input elements. The number of each element also represents the order in the input matrix. ( B) A dendrogram drawn using the default heuristics in R. The branches in the dendrogram are labeled from “a” to “i” in the order in which clusters are merged. ( C) A dendrogram reordered using MOLO with the smallest distance. The global structures in a shape of the right triangle are highlighted.

Figure 3.. The recursive algorithm for ordering a dendrogram structure based on the minimum distance.

Figure 4.. Comparison of dendrograms from different linkage algorithms using R’s default ordering heuristics.

The element 32 and 34 are highlighted.

Figure 5.. Comparison of dendrograms from different linkage algorithms after applying the MOLO method based on the smallest distance.

The element 32 and 34 are highlighted.

Figure 6.. Comparison of leaf ordering methods in cluster heat maps.

The default hierarchical clustering (HC), the Gruvaeus and Wainer’s method (GW), the optimal leaf ordering (OLO), and the MOLO method are applied to the Fisher’s Iris data set.

Figure 7.. Cluster heat map of the data matrix after applying the MOLO method based on the smallest distance.

Figure 8.. Comparison of dendrogram structures resulting from different leaf ordering methods.

The rows from the example data sets are shown.

Figure 9.. Cluster heat map of the data matrix after applying the MOLO method based on the average distance.

The rows and columns with an inverse relationship are highlighted in the dendrograms.

Figure 10.. Comparison of dendrogram structures resulting from different leaf ordering methods in a limited display space.

The rows from the example data sets are shown.