Abstract:Structural magnetic resonance imaging (sMRI) provides accurate estimates of the brain's structural organization and learning invariant brain representations from sMRI is an enduring issue in neuroscience. Previous deep representation learning models ignore the fact that the brain, as the core of human cognitive activity, is distinct from other organs whose primary attribute is anatomy. Therefore, capturing the semantic structure that dominates interindividual cognitive variability is key to accurately representing the brain. Given that this high-level semantic information is subtle, distributed, and interdependently latent in the brain structure, sMRI-based models need to capture fine-grained details and understand how they relate to the overall global structure. However, existing models are optimized by simple objectives, making features collapse into homogeneity and worsening simultaneous representation of fine-grained information and holistic semantics, causing a lack of biological plausibility and interpretation of cognition. Here, we propose MCIAT, a unified framework that combines Multi-task Collaborative pre-training and Individual-Adaptive-Tokens fine-tuning. Specifically, we first synthesize restorative learning, age prediction auxiliary learning and adversarial learning as a joint proxy task for deep semantic representation learning. Then, a mutual-attention-based token selection method is proposed to highlight discriminative features. The proposed MCIAT achieves state-of-the-art diagnosis performance on the ADHD-200 dataset compared with several sMRI-based approaches and shows superior generalization on the MCIC and OASIS datasets. Moreover, we studied 12 behavioral tasks and found significant associations between cognitive functions and MCIAT-established representations, which verifies the interpretability of our proposed framework.

A novel serial deep multi-task learning model for large scale biomedical semantic indexing

Biomedical Semantic Indexing by Deep Neural Network with Multi-Task Learning

Deep Semisupervised Multitask Learning Model and Its Interpretability for Survival Analysis.

Large-Scale Online Semantic Indexing of Biomedical Articles via an Ensemble of Multi-Label Classification Models

Semantic Indexing with Deep Learning: a Case Study

Biomedical Literature Classification With A Cnns-Based Hybrid Learning Network

Multi-task Transfer Learning for Biomedical Machine Reading Comprehension.

Hierarchical Multi-layer Transfer Learning Model for Biomedical Question Answering

Multitask Learning for Biomedical Named Entity Recognition with Cross-Sharing Structure

Cross-type Biomedical Named Entity Recognition with Deep Multi-Task Learning

Language model based on deep learning network for biomedical named entity recognition

A Multi-Task Approach for Improving Biomedical Named Entity Recognition by Incorporating Multi-Granularity Information.

Deep scaled dot-product attention based domain adaptation model for biomedical question answering

A Multichannel Biomedical Named Entity Recognition Model Based on Multitask Learning and Contextualized Word Representations

Multi-task deep learning for medical image computing and analysis: A review

Semantics-Depth-Symbiosis: Deeply Coupled Semi-Supervised Learning of Semantics and Depth

Biomedical Argument Mining Based on Sequential Multi-Task Learning

Multi-probe Attention Neural Network for COVID-19 Semantic Indexing

Multi-task Collaborative Pre-training and Individual-adaptive-tokens Fine-tuning: A Unified Framework for Brain Representation Learning

Hierarchical shared transfer learning for biomedical named entity recognition

Dataset-aware multi-task learning approaches for biomedical named entity recognition