The impact of quality assurance assessment on diffusion tensor imaging outcomes in a large-scale population-based cohort

Abstract

Background: Diffusion tensor imaging (DTI) is applied in investigation of brain biomarkers for neurodevelopmental and neurodegenerative disorders. However, the quality of DTI measurements, like other neuroimaging techniques, is susceptible to several confounding factors (e.g., motion, eddy currents), which have only recently come under scrutiny. These confounds are especially relevant in adolescent samples where data quality may be compromised in ways that confound interpretation of maturation parameters. The current study aims to leverage DTI data from the Philadelphia Neurodevelopmental Cohort (PNC), a sample of 1601 youths with ages of 8-21 who underwent neuroimaging, to: 1) establish quality assurance (QA) metrics for the automatic identification of poor DTI image quality; 2) examine the performance of these QA measures in an external validation sample; 3) document the influence of data quality on developmental patterns of typical DTI metrics.

Methods: All diffusion-weighted images were acquired on the same scanner. Visual QA was performed on all subjects completing DTI; images were manually categorized as Poor, Good, or Excellent. Four image quality metrics were automatically computed and used to predict manual QA status: Mean voxel intensity outlier count (MEANVOX), Maximum voxel intensity outlier count (MAXVOX), mean relative motion (MOTION) and temporal signal-to-noise ratio (TSNR). Classification accuracy for each metric was calculated as the area under the receiver-operating characteristic curve (AUC). A threshold was generated for each measure that best differentiated visual QA status and applied in a validation sample. The effects of data quality on sensitivity to expected age effects in this developmental sample were then investigated using the traditional MRI diffusion metrics: fractional anisotropy (FA) and mean diffusivity (MD). Finally, our method of QA is compared with DTIPrep.

Results: TSNR (AUC=0.94) best differentiated Poor data from Good and Excellent data. MAXVOX (AUC=0.88) best differentiated Good from Excellent DTI data. At the optimal threshold, 88% of Poor data and 91% Good/Excellent data were correctly identified. Use of these thresholds on a validation dataset (n=374) indicated high accuracy. In the validation sample 83% of Poor data and 94% of Excellent data was identified using thresholds derived from the training sample. Both FA and MD were affected by the inclusion of poor data in an analysis of an age, sex and race matched comparison sample. In addition, we show that the inclusion of poor data results in significant attenuation of the correlation between diffusion metrics (FA and MD) and age during a critical neurodevelopmental period. We find higher correspondence between our QA method and DTIPrep for Poor data, but we find our method to be more robust for apparently high-quality images.

Conclusion: Automated QA of DTI can facilitate large-scale, high-throughput quality assurance by reliably identifying both scanner and subject induced imaging artifacts. The results present a practical example of the confounding effects of artifacts on DTI analysis in a large population-based sample, and suggest that estimates of data quality should not only be reported but also accounted for in data analysis, especially in studies of development.

Keywords: Adolescence; Automated quality assurance; Brain maturation; Diffusion tensor imaging; Motion.

Figure 1

Initial enrollment in the Philadelphia Neurodevelopmental Cohort and a count and explanation of excluded DTI data.

Figure 2

Two examples of Poor DTI data from the PNC. A. An example of image striping likely cause by sub-optimal gradient performance. B. An example of data with inter-slice and intra-slice signal drop-out likely caused by the interaction of subject motion and diffusion encoding.

Figure 3

DTI Preprocessing Pipeline. 64 direction DTI data was collected in two consecutive series (A) and merged into a single time series (B). Automated quality assurance was performed (C). A field map (D) was acquired and used in distortion correction. DTI images were corrected for motion and eddy currents (E). Last, a tensor model was fit (F).

Figure 4

Receiver operator characteristic curves of TSNR, MAXVOX, MEANVOX, and MOTION. (A) ROCs for differentiating Poor data from Acceptable data (Good+Excellent). TSNR best differentiated Poor data from Acceptable data. (B) ROCs for differentiating Excellent data from Good data. MAXVOX best differentiated Excellent data from Good Data.

Figure 5

Temporal signal-to-noise ratio (TSNR) maps for each QA group (matched sub-sample: n=146 per group).

Figure 6

Statistical differences (z-maps) in TSNR between data passing QA (Good+Excellent) in comparison to those failing QA (Poor data). A. Z-maps indicating higher TSNR across the brain in data passing QA as compared to data failing QA B. Data failing QA shows limited regions where TSNR is higher than data passing QA. This difference is limited to the corpus callosum, a highly anisotropic region, where TSNR tends to be low in all individuals. C. Quantified regional TSNR in gray matter ROIs (Harvard-Oxford atlas) in data passing and failing QA. Data passing QA had significantly higher TSNR in all ROIs. D. Quantified regional TSNR in white matter ROIs (JHU atlas) in data passing and failing QA. Data passing QA had significantly higher TSNR in all ROIs except the corticospinal tract.

Figure 7

Statistical difference in FA and MD between data passing QA (Good+Excellent) in comparison to those failing QA (Poor data). (A) FA was significantly higher throughout white matter in higher quality data while (B) MD was higher in Poor data, particularly at the edge of white matter tracts.

Figure 8

Normalized (z-transformed) regional (A) FA and (B) MD values in Poor data. ROI data from ICBM-JHU White Matter Atlas. Data are normalized data that passes QA.

Figure 9

Pearson correlation coefficients between Age and FA in data failing QA (Poor) and data passing QA (Good+Excellent). A. Data that failed QA had a significantly lower correlation between age and FA as compared to data that passed QA. B. Data that failed QA had a significantly lower correlation between age and MD as compared to data that passed QA. C & D: Correlation between FA and age in males and females in data passing QA. E & F: Correlation between FA and age in males and females in data failing QA. G & H. Summary plot of the correlation between age and DTI metric in data failing QA, passing QA and all data combined. The striped bar shows Pearson correlation when all data is combined. Estimating this association is strongest when data of low quality is excluded from analysis. *p<.05 as compared to Failed QA or All.

Figure 10

Comparison of DTIPrep QA with our Visual QA methods. The two methods were in the highest agreement for Poor data followed by Good then Excellent. However, DTIPrep at the default setting considered more data to fail.

Figure 11

Pearson correlation coefficients between Age and FA in data failing visual QA (Poor), data passing visual QA (Good+Excellent) and data Passing DTIPrep QA. Data passing visual QA, but not DTIPrep QA, had a significantly higher correlation between Age and FA. *p<0.05 as compared to Failed Visual QA.