Abstract:We use test–retest data from three public data sets to determine within‐version reliability and between‐version compatibility across 42 regional outputs from FreeSurfer version 7.1, 6.0, and 5.3. Overall, we find generally high within‐version reliability across most versions, however, considerable differences are observed when analyzing between‐version compatibility for regional cortical thickness, surface area, and subcortical volumes. Automatic neuroimaging processing tools provide convenient and systematic methods for extracting features from brain magnetic resonance imaging scans. One tool, FreeSurfer, provides an easy‐to‐use pipeline to extract cortical and subcortical morphometric measures. There have been over 25 stable releases of FreeSurfer, with different versions used across published works. The reliability and compatibility of regional morphometric metrics derived from the most recent version releases have yet to be empirically assessed. Here, we used test–retest data from three public data sets to determine within‐version reliability and between‐version compatibility across 42 regional outputs from FreeSurfer versions 7.1, 6.0, and 5.3. Cortical thickness from v7.1 was less compatible with that of older versions, particularly along the cingulate gyrus, where the lowest version compatibility was observed (intraclass correlation coefficient 0.37–0.61). Surface area of the temporal pole, frontal pole, and medial orbitofrontal cortex, also showed low to moderate version compatibility. We confirm low compatibility between v6.0 and v5.3 of pallidum and putamen volumes, while those from v7.1 were compatible with v6.0. Replication in an independent sample showed largely similar results for measures of surface area and subcortical volumes, but had lower overall regional thickness reliability and compatibility. Batch effect correction may adjust for some inter‐version effects when most sites are run with one version, but results vary when more sites are run with different versions. Age associations in a quality controlled independent sample (N = 106) revealed version differences in results of downstream statistical analysis. We provide a reference to highlight the regional metrics that may yield recent version‐related inconsistencies in published findings. An interactive viewer is provided at http://data.brainescience.org/Freesurfer_Reliability/.

SU-E-I-70: Comparison of Brain Volume Quantification with NeuroQuant & FreeSurfer

Comparison of Automated Brain Volume Measures obtained with NeuroQuant® and FreeSurfer

Evaluating brain volume segmentation accuracy and reliability of FreeSurfer and Neurophet AQUA at variations in MRI magnetic field strengths

Comparison of Automated Brain Volume Measures by NeuroQuant vs. Freesurfer in Patients with Mild Cognitive Impairment: Effect of Slice Thickness

Abstract WP292: Quantitative Susceptibility Mapping for Reliably Measuring Hematoma Volume in Magnetic Resonance Imaging

Agreement and Reliability between Clinically Available Software Programs in Measuring Volumes and Normative Percentiles of Segmented Brain Regions

Comparison of Inter-Method Agreement and Reliability for Automatic Brain Volumetry Using Three Different Clinically Available Software Packages

Reliability of automated brain volumetric analysis: A test by comparing NeuroQuant and volBrain software

Comparison of Manual and Automatic Methods of Hippocampus Segmentation

Comparing Automated Subcortical Volume Estimation Methods; Amygdala Volumes Estimated by FSL and FreeSurfer Have Poor Consistency

Reproducibility of volume and asymmetry measurements of hippocampus, amygdala, and entorhinal cortex on traveling volunteers: a multisite MP2RAGE prospective study

Inter- and Intra-Scanner Variability of Automated Brain Volumetry on Three Magnetic Resonance Imaging Systems in Alzheimer's Disease and Controls

Quantifying white matter hyperintensity and brain volumes in heterogeneous clinical and low-field portable MRI

Multisite test–retest reliability and compatibility of brain metrics derived from FreeSurfer versions 7.1, 6.0, and 5.3

Brain regional volume estimations with NeuroQuant and FIRST: a study in patients with a clinically isolated syndrome

Quantifying numerical and spatial reliability of hippocampal and amygdala subdivisions in FreeSurfer

Evaluating Traditional, Deep Learning, and Subfield Methods for Automatically Segmenting the Hippocampus from MRI

Development and validation of a deep learning-based automatic segmentation model for assessing intracranial volume: comparison with NeuroQuant, FreeSurfer, and SynthSeg

Quantitative Accuracy Assessment of the NeuroEXPLORER for Diverse Imaging Applications: Moving Beyond Standard Evaluations

Improving segmentation reliability of multi-scanner brain images using a generative adversarial network

Brain Parcellation Repeatability and Reproducibility Using Conventional and Quantitative 3D MR Imaging