Abstract:We develop a novel harmonization approach for T1‐weighted magnetic resonance imaging using a style‐encoding generative adversarial network that can be used to harmonize entire images for a variety of international, multi‐cohort, neuroimaging collaborations. Results demonstrated that this model avoids the need to control for clinical or demographic information. We showed that our harmonization removed the cross‐site variances, while preserving the anatomical information and clinical meaningful patterns. Recent work within neuroimaging consortia have aimed to identify reproducible, and often subtle, brain signatures of psychiatric or neurological conditions. To allow for high‐powered brain imaging analyses, it is often necessary to pool MR images that were acquired with different protocols across multiple scanners. Current retrospective harmonization techniques have shown promise in removing site‐related image variation. However, most statistical approaches may over‐correct for technical, scanning‐related, variation as they cannot distinguish between confounded image‐acquisition based variability and site‐related population variability. Such statistical methods often require that datasets contain subjects or patient groups with similar clinical or demographic information to isolate the acquisition‐based variability. To overcome this limitation, we consider site‐related magnetic resonance (MR) imaging harmonization as a style transfer problem rather than a domain transfer problem. Using a fully unsupervised deep‐learning framework based on a generative adversarial network (GAN), we show that MR images can be harmonized by inserting the style information encoded from a single reference image, without knowing their site/scanner labels a priori. We trained our model using data from five large‐scale multisite datasets with varied demographics. Results demonstrated that our style‐encoding model can harmonize MR images, and match intensity profiles, without relying on traveling subjects. This model also avoids the need to control for clinical, diagnostic, or demographic information. We highlight the effectiveness of our method for clinical research by comparing extracted cortical and subcortical features, brain‐age estimates, and case–control effect sizes before and after the harmonization. We showed that our harmonization removed the site‐related variances, while preserving the anatomical information and clinical meaningful patterns. We further demonstrated that with a diverse training set, our method successfully harmonized MR images collected from unseen scanners and protocols, suggesting a promising tool for ongoing collaborative studies. Source code is released in USC‐IGC/style_transfer_harmonization (github.com).

Is Your Style Transfer Doing Anything Useful? An Investigation Into Hippocampus Segmentation and the Role of Preprocessing

Evaluating Traditional, Deep Learning, and Subfield Methods for Automatically Segmenting the Hippocampus from MRI

Style transfer generative adversarial networks to harmonize multisite MRI to a single reference image to avoid overcorrection

Neural Style Transfer Improves 3D Cardiovascular MR Image Segmentation on Inconsistent Data

Hippocampus segmentation in magnetic resonance images of Alzheimer's patients using Deep machine learning

Comparative assessment of established and deep learning‐based segmentation methods for hippocampal volume estimation in brain magnetic resonance imaging analysis

Fully Automated Hippocampus Segmentation using T2-informed Deep Convolutional Neural Networks

Geodesic shape regression based deep learning segmentation for assessing longitudinal hippocampal atrophy in dementia progression

Hippocampus Substructure Segmentation Using Morphological Vision Transformer Learning

Mitigating analytical variability in fMRI results with style transfer

Feature Selection based on Machine Learning in MRIs for Hippocampal Segmentation

Symmetric diffeomorphic image registration with cross-correlation: Evaluating automated labeling of elderly and neurodegenerative brain

Hippocampus segmentation on epilepsy and Alzheimer's disease studies with multiple convolutional neural networks

An automated, geometry-based method for hippocampal shape and thickness analysis

Surface-Based Morphometric Analysis of Hippocampal Subfields in Mild Cognitive Impairment and Alzheimer's Disease.

Automated quality assessment using appearance-based simulations and hippocampus segmentation on low-field paediatric brain MR images

Extended 2D Consensus Hippocampus Segmentation

Hippocampal shape analysis of Alzheimer disease based on machine learning methods.

Robust machine learning segmentation for large-scale analysis of heterogeneous clinical brain MRI datasets

A scalable method to improve gray matter segmentation at ultra high field MRI

Hippocampal Segmentation in Brain MRI Images Using Machine Learning Methods: A Survey