A filtering method to generate high quality short reads using illumina paired-end technology

Abstract

Consensus between independent reads improves the accuracy of genome and transcriptome analyses, however lack of consensus between very similar sequences in metagenomic studies can and often does represent natural variation of biological significance. The common use of machine-assigned quality scores on next generation platforms does not necessarily correlate with accuracy. Here, we describe using the overlap of paired-end, short sequence reads to identify error-prone reads in marker gene analyses and their contribution to spurious OTUs following clustering analysis using QIIME. Our approach can also reduce error in shotgun sequencing data generated from libraries with small, tightly constrained insert sizes. The open-source implementation of this algorithm in Python programming language with user instructions can be obtained from https://github.com/meren/illumina-utils.

Figure 1. Structure of V6 fusion primers used to generate amplicon libraries for Illumina sequencing.

Figure 2. Comparison of three filtering methods.

The top panel shows the ratio of pairs identified as low quality versus all pairs analyzed for each method. The total number of pairs in each dataset is shown in the bottom panel.

Figure 3. Paired-end reads from 33 samples that passed and failed the quality filtering by individual methods are compared in Venn diagrams.

The mean quality scores of paired-end reads from the numbered regions in Venn diagrams are shown below. In each panel, the top and bottom lines show read 1 and read 2, respectively. The mean quality of each pair at each nucleotide position is also shown with a smooth line.

Figure 4. Comparison of the relative cluster counts and the absolute number of clusters identified in reads filtered by three different methods.

The top panel shows the relative number of 97% OTUs for each method, with the method that produced the largest number assigned a value of 1.0. The bottom panel presents the actual number of OTUs for each method.