Spectral Jaccard Similarity: A New Approach to Estimating Pairwise Sequence Alignments

Tavor Z Baharav; Govinda M Kamath; David N Tse; Ilan Shomorony

doi:10.1016/j.patter.2020.100081

Spectral Jaccard Similarity: A New Approach to Estimating Pairwise Sequence Alignments

Patterns (N Y). 2020 Jul 31;1(6):100081. doi: 10.1016/j.patter.2020.100081. eCollection 2020 Sep 11.

Authors

Tavor Z Baharav¹, Govinda M Kamath², David N Tse¹, Ilan Shomorony³

Affiliations

¹ Department of Electrical Engineering, Stanford University, Stanford, CA 94305, USA.
² Microsoft Research New England, Cambridge, MA 02142, USA.
³ Department of Electrical and Computer Engineering, University of Illinois, Urbana-Champaign, IL 61801, USA.

Abstract

Pairwise sequence alignment is often a computational bottleneck in genomic analysis pipelines, particularly in the context of third-generation sequencing technologies. To speed up this process, the pairwise k-mer Jaccard similarity is sometimes used as a proxy for alignment size in order to filter pairs of reads, and min-hashes are employed to efficiently estimate these similarities. However, when the k-mer distribution of a dataset is significantly non-uniform (e.g., due to GC biases and repeats), Jaccard similarity is no longer a good proxy for alignment size. In this work, we introduce a min-hash-based approach for estimating alignment sizes called Spectral Jaccard Similarity, which naturally accounts for uneven k-mer distributions. The Spectral Jaccard Similarity is computed by performing a singular value decomposition on a min-hash collision matrix. We empirically show that this new metric provides significantly better estimates for alignment sizes, and we provide a computationally efficient estimator for these spectral similarity scores.

Keywords: DNA sequencing; Jaccard similarity; genome assembly; locality-sensitive hashing; min-hash; sequence alignment; singular value decomposition; spectral methods.