Reliability of structural MRI measurements: The effects of scan session, head tilt, inter-scan interval, acquisition sequence, FreeSurfer version and processing stream

Abstract

Background: Large-scale longitudinal and multi-centre studies are used to explore neuroimaging markers of normal ageing, and neurodegenerative and mental health disorders. Longitudinal changes in brain structure are typically small, therefore the reliability of automated techniques is crucial. Determining the effects of different factors on reliability allows investigators to control those adversely affecting reliability, calculate statistical power, or even avoid particular brain measures with low reliability. This study examined the impact of several image acquisition and processing factors and documented the test-retest reliability of structural MRI measurements.

Methods: In Phase I, 20 healthy adults (11 females; aged 20-30 years) were scanned on two occasions three weeks apart on the same scanner using the ADNI-3 protocol. On each occasion, individuals were scanned twice (repetition), after re-entering the scanner (reposition) and after tilting their head forward. At one year follow-up, nine returning individuals and 11 new volunteers were recruited for Phase II (11 females; aged 22-31 years). Scans were acquired on two different scanners using the ADNI-2 and ADNI-3 protocols. Structural images were processed using FreeSurfer (v5.3.0, 6.0.0 and 7.1.0) to provide subcortical and cortical volume, cortical surface area and thickness measurements. Intra-class correlation coefficients (ICC) were calculated to estimate test-retest reliability. We examined the effect of repetition, reposition, head tilt, time between scans, MRI sequence and scanner on reliability of structural brain measurements. Mean percentage differences were also calculated in supplementary analyses.

Results: Using the FreeSurfer v7.1.0 longitudinal pipeline, we observed high reliability for subcortical and cortical volumes, and cortical surface areas at repetition, reposition, three weeks and one year (mean ICCs>0.97). Cortical thickness reliability was lower (mean ICCs>0.82). Head tilt had the greatest adverse impact on ICC estimates, for example reducing mean right cortical thickness to ICC=0.74. In contrast, changes in ADNI sequence or MRI scanner had a minimal effect. We observed an increase in reliability for updated FreeSurfer versions, with the longitudinal pipeline consistently having a higher reliability than the cross-sectional pipeline.

Discussion: Longitudinal studies should monitor or control head tilt to maximise reliability. We provided the ICC estimates and mean percentage differences for all FreeSurfer brain regions, which may inform power analyses for clinical studies and have implications for the design of future longitudinal studies.

Keywords: Longitudinal; Morphology; Reproducibility; Structural MRI.

Fig. 1

Illustration of the structural MRI data acquisition process in Phase I and II of the PIN study. In Phase II, the order of visits was balanced for participants so the D1 – D3 scans (on Scanner 2) were acquired first for ten participants. C3 and D3 scans were acquired using a different MRI sequence.

Fig. 2

ICC values of subcortical and cortical volumes, cortical thickness, and cortical surface area morphometric measurements for ‘head tilt’ comparison from FreeSurfer 7.1.0 longitudinal stream. This change in pitch has been reported as the most common type of head movement seen in adult groups during an MRI scan (Cusack et al., 2017). For cortical thickness measurements (Image C), 16 regions had an ICC<0.7 (minimum value = 0.402) (see Supplementary Materials S2).

Fig. 3

ICC values of subcortical and cortical volumes, cortical thickness, and cortical surface area morphometric measurements for ‘three-week’ comparison from FreeSurfer 7.1.0 longitudinal stream. For cortical thickness measurements (Image C), two regions had an ICC<0.7 (minimum value = 0.598) (see Supplementary Materials S2).

Fig. 4

Left: scatterplot of right superior parietal thickness (mm) showing the effect of head tilt (A4 scan), which is associated with relatively low reliability (ICC=0.40) and an apparent reduction in cortical thickness. Right: scatterplot for volume (mm³) comparison of the same region is shown where there is higher reliability (ICC=0.96).

Fig. 5

Statistical power of ‘three-week’ and ‘head tilt’ comparisons across ten regional thicknesses of the highest and lowest reliability. Graph displays the sample size needed to detect longitudinal changes in thickness at p<0.05 significance level and 1-β statistical power = 0.80 (two-tailed). For example, for a study to be sufficiently powered to detect a 1% longitudinal change in the left parahippocampal thickness, 36 participants would be required using data from the three-week scan. However, 50 participants would be needed if there was a change in head tilt.