Nonpareil 3: Fast Estimation of Metagenomic Coverage and Sequence Diversity

Abstract

Estimations of microbial community diversity based on metagenomic data sets are affected, often to an unknown degree, by biases derived from insufficient coverage and reference database-dependent estimations of diversity. For instance, the completeness of reference databases cannot be generally estimated since it depends on the extant diversity sampled to date, which, with the exception of a few habitats such as the human gut, remains severely undersampled. Further, estimation of the degree of coverage of a microbial community by a metagenomic data set is prohibitively time-consuming for large data sets, and coverage values may not be directly comparable between data sets obtained with different sequencing technologies. Here, we extend Nonpareil, a database-independent tool for the estimation of coverage in metagenomic data sets, to a high-performance computing implementation that scales up to hundreds of cores and includes, in addition, a k-mer-based estimation as sensitive as the original alignment-based version but about three hundred times as fast. Further, we propose a metric of sequence diversity (N_d ) derived directly from Nonpareil curves that correlates well with alpha diversity assessed by traditional metrics. We use this metric in different experiments demonstrating the correlation with the Shannon index estimated on 16S rRNA gene profiles and show that N_d additionally reveals seasonal patterns in marine samples that are not captured by the Shannon index and more precise rankings of the magnitude of diversity of microbial communities in different habitats. Therefore, the new version of Nonpareil, called Nonpareil 3, advances the toolbox for metagenomic analyses of microbiomes. IMPORTANCE Estimation of the coverage provided by a metagenomic data set, i.e., what fraction of the microbial community was sampled by DNA sequencing, represents an essential first step of every culture-independent genomic study that aims to robustly assess the sequence diversity present in a sample. However, estimation of coverage remains elusive because of several technical limitations associated with high computational requirements and limiting statistical approaches to quantify diversity. Here we described Nonpareil 3, a new bioinformatics algorithm that circumvents several of these limitations and thus can facilitate culture-independent studies in clinical or environmental settings, independent of the sequencing platform employed. In addition, we present a new metric of sequence diversity based on rarefied coverage and demonstrate its use in communities from diverse ecosystems.

Keywords: bioinformatics; coverage; metagenomics; microbial ecology.

FIG 1

Speedup per number of processors in Nonpareil. Nonpareil estimations of the LL_1007B data set (3) were performed with alignment kernel and default parameters in multiple processors of a node (light blue), a single processor of multiple nodes (pink), and four processors of multiple nodes (dark green). The base time (one processor of one node) was 82.98 h.

FIG 2

Comparison of Nonpareil N_d sequence diversity and 16S rRNA gene OTU Shannon H′ taxonomic diversity indices on 90 metagenomes. Each data point represents the estimates on N_d (x axis) and H′ (y axis). The y-axis value of each point indicates the Bayesian analysis-corrected Shannon index, and the line extending from the low part of each data point represents the exact observed (maximum-likelihood) Shannon index. The color of each point indicates the type of biome of each data set, the shape indicates the sequencing platform, and the size indicates the estimated coverage of the 16S rRNA gene profile (Turing-Good estimate). For each biome, the IQR of both estimates is represented as semitransparent rectangles. The least-squares linear correlation model is represented in gray, including the central estimate (solid line), the 95% confidence interval (dashed-line band), and the 80 and 95% prediction intervals (dotted-line bands). Labeled data sets fell outside the 80% prediction interval. The inset shows the residuals from the linear model against the Turing-Good estimate of 16S rRNA gene coverage.