Abstract:Motivation: Recent brain mapping efforts are producing large-scale whole-brain images using different imaging modalities. Accurate alignment and delineation of anatomical structures in these images are essential for numerous studies. These requirements are typically modeled as two distinct tasks: Registration and segmentation. However, prevailing methods, fail to fully explore and utilize the inherent correlation and complementarity between the two tasks. Furthermore, variations in brain anatomy, brightness and texture pose another formidable challenge in designing multi-modal similarity metrics. A high-throughput approach capable of overcomingthe bottleneck of multi-modal similarity metric design, while effective leveraging the highly correlated and complementary nature of two tasks is highly desirable. Results: We introduce a deep learning framework for joint registration and segmentation of multi-modal brain images. Under this framework, registration and segmentation tasks are deeply coupled and collaborated at two hierarchical layers. In the inner layer, we establish a strong feature-level coupling between the two tasks by learning a unified common latent feature representation. In the outer layer, we introduce a mutually supervised dual-branch network to decouple latent features and facilitate task-level collaboration between registration and segmentation. Since the latent features we designed are also modality-independent, the bottleneck of designing multi-modal similarity metric is essentially addressed. Another merit offered by this framework is the interpretability of latent features, which allows intuitive manipulation of feature learning, thereby further enhancing network training efficiency and the performance of both tasks. Extensive experiments conducted on both multi-modal and mono-modal datasets of mouse and human brains demonstrate the superiority of our method. Availability: The code is available at https://github.com/tingtingup/DCRS. Supplementary information: Supplementary data are available at Bioinformaticsonline.

Learning Task-Specific Morphological Representation for Pyramidal Cells via Mutual Information Minimization

Towards Understanding Multi-Task Learning (Generalization) of LLMs via Detecting and Exploring Task-Specific Neurons

Learning meaningful representation of single-neuron morphology via large-scale pre-training

Multi-Task Collaborative Pre-Training and Adaptive Token Selection: A Unified Framework for Brain Representation Learning

Deep Mining of Subtle Differences in Cell Morphology Via Deep Learning

Graph-based cell pattern recognition for merging the multi-modal optical microscopic image of neurons

Multi-task Collaborative Pre-training and Individual-adaptive-tokens Fine-tuning: A Unified Framework for Brain Representation Learning

Deep Multi-Task and Task-Specific Feature Learning Network for Robust Shape Preserved Organ Segmentation.

Generalizable Morphological Profiling of Cells by Interpretable Unsupervised Learning

Multi-level feature fusion network for neuronal morphology classification

A Framework for Deep Multitask Learning With Multiparametric Magnetic Resonance Imaging for the Joint Prediction of Histological Characteristics in Breast Cancer

A Dual-Pathway Network Combining Global and Local Representation Enhanced Automatic Segmentation of Hippocampus

Learning deep representation of multityped objects and tasks

Multiple Instance Learning with Task-Specific Multi-Level Features for Weakly Annotated Histopathological Image Classification

Brain Image Segmentation for Ultrascale Neuron Reconstruction via an Adaptive Dual-Task Learning Network

Deep coupled registration and segmentation of multimodal whole-brain images

MOLE: MOdular Learning FramEwork via Mutual Information Maximization

Hierarchical Deep Multi-task Learning with Attention Mechanism for Similarity Learning

Deep residual contextual and subpixel convolution network for automated neuronal structure segmentation in micro-connectomics

Multi-task Learning with Deformable Convolution

Multi-task Learning for Macromolecule Classification, Segmentation and Coarse Structural Recovery in Cryo-Tomography