Abstract:Motivation: Recent brain mapping efforts are producing large-scale whole-brain images using different imaging modalities. Accurate alignment and delineation of anatomical structures in these images are essential for numerous studies. These requirements are typically modeled as two distinct tasks: Registration and segmentation. However, prevailing methods, fail to fully explore and utilize the inherent correlation and complementarity between the two tasks. Furthermore, variations in brain anatomy, brightness and texture pose another formidable challenge in designing multi-modal similarity metrics. A high-throughput approach capable of overcomingthe bottleneck of multi-modal similarity metric design, while effective leveraging the highly correlated and complementary nature of two tasks is highly desirable. Results: We introduce a deep learning framework for joint registration and segmentation of multi-modal brain images. Under this framework, registration and segmentation tasks are deeply coupled and collaborated at two hierarchical layers. In the inner layer, we establish a strong feature-level coupling between the two tasks by learning a unified common latent feature representation. In the outer layer, we introduce a mutually supervised dual-branch network to decouple latent features and facilitate task-level collaboration between registration and segmentation. Since the latent features we designed are also modality-independent, the bottleneck of designing multi-modal similarity metric is essentially addressed. Another merit offered by this framework is the interpretability of latent features, which allows intuitive manipulation of feature learning, thereby further enhancing network training efficiency and the performance of both tasks. Extensive experiments conducted on both multi-modal and mono-modal datasets of mouse and human brains demonstrate the superiority of our method. Availability: The code is available at https://github.com/tingtingup/DCRS. Supplementary information: Supplementary data are available at Bioinformaticsonline.

Developing Brain Atlas through Deep Learning

Automated 3D Fetal Brain Segmentation Using an Optimized Deep Learning Approach

NeuroAtlas: An Artificial Intelligence-based Framework for Annotation, Segmentation and Registration of Large Scale Biomedical Imaging Data

Towards segmentation and spatial alignment of the human embryonic brain using deep learning for atlas-based registration

Unsupervised Segmentation of Fetal Brain MRI using Deep Learning Cascaded Registration

MesoNet allows automated scaling and segmentation of mouse mesoscale cortical maps using machine learning

NeuReg: Domain-invariant 3D Image Registration on Human and Mouse Brains

Two-Stage Deep Learning Segmentation for Tiny Brain Regions.

Exploring Brain-wide Development of Inhibition through Deep Learning

Subcortical Brain Segmentation Based on Atlas Registration and Linearized Kernel Sparse Representative Classifier

Deep coupled registration and segmentation of multimodal whole-brain images

BrainSegFounder: Towards 3D foundation models for neuroimage segmentation

A deep learning model for brain segmentation across pediatric and adult populations

A next-generation, histological atlas of the human brain and its application to automated brain MRI segmentation

Bi-channel image registration and deep-learning segmentation (BIRDS) for efficient, versatile 3D mapping of mouse brain

DeepMRSeg: A convolutional deep neural network for anatomy and abnormality segmentation on MR images

Generalizing deep learning brain segmentation for skull removal and intracranial measurements.

Brain MR Atlas Construction Using Symmetric Deep Neural Inpainting

Integrated 3d flow-based multi-atlas brain structure segmentation.

DARTS: DenseUnet-based Automatic Rapid Tool for brain Segmentation

DeepAtlas: Joint Semi-Supervised Learning of Image Registration and Segmentation