Abstract:Schizophrenia (SZ) is a severe brain disorder marked by diverse cognitive impairments, abnormalities in brain structure, function, and genetic factors. Its complex symptoms and overlap with other psychiatric conditions challenge traditional diagnostic methods, necessitating advanced systems to improve precision. Existing research studies have mostly focused on imaging data, such as structural and functional MRI, for SZ diagnosis. There has been less focus on the integration of genomic features despite their potential in identifying heritable SZ traits. In this study, we introduce a Multi-modal Imaging Genomics Transformer (MIGTrans), that attentively integrates genomics with structural and functional imaging data to capture SZ-related neuroanatomical and connectome abnormalities. MIGTrans demonstrated improved SZ classification performance with an accuracy of 86.05% (+/- 0.02), offering clear interpretations and identifying significant genomic locations and brain morphological/connectivity patterns associated with SZ.

What problem does this paper attempt to address?

The problem this paper attempts to address is improving the diagnostic accuracy of schizophrenia (SZ) by integrating multi-modal data, including structural imaging, functional connectomics, and genomic features. Specifically, existing studies mainly focus on imaging data (such as structural and functional MRI), while the integration of genomic features is relatively less explored, despite their potential in identifying hereditary schizophrenia traits. Therefore, this study introduces a method called Multi-modal Imaging Genomics Transformer (MIGTrans), which aims to effectively fuse genomic data with structural and functional imaging data through the self-attention mechanism, thereby capturing neuroanatomical and connectomic abnormalities associated with schizophrenia. ### Main Objectives: 1. **Improve the classification performance of schizophrenia**: Enhance the diagnostic accuracy of schizophrenia by integrating multi-modal data, especially genomic data. 2. **Provide clinical interpretability**: Identify significant gene loci and brain morphology/connectivity patterns associated with schizophrenia to support clinical decision-making. 3. **Explore complementary information of multi-modal data**: Utilize the complementary information between genomic, connectomic, and structural imaging data to reveal new biological pathways. ### Specific Methods: - **Genomic Encoder**: Learn the nonlinear representation of genomic features through neural networks. - **Connectomic Encoder**: Learn the nonlinear representation of functional connectomics through neural networks. - **Structural MRI Encoder**: Use a pre-trained 3D DenseNet121 to extract morphological features of structural imaging and learn attention in spatial and channel dimensions through the Spatial Sequence Attention (SSA) mechanism. - **Fusion Transformer**: Gradually integrate genomic, connectomic, and structural imaging features through the Cross-modal Multi-Head Attention (x-MHA) mechanism. ### Experimental Results: - **Quantitative Evaluation**: MIGTrans performed excellently in the schizophrenia classification task, achieving an accuracy of 86.05%, outperforming baseline methods. - **Qualitative Evaluation**: Through the visualization of attention maps and functional brain networks, significant brain regions and connectivity patterns associated with schizophrenia were demonstrated, validating the model's effectiveness and clinical interpretability. ### Conclusion: MIGTrans effectively combines genomic, connectomic, and structural imaging data through a novel three-step self-attention integration method, improving the diagnostic accuracy of schizophrenia and providing valuable clinical insights. This offers new tools and methods for schizophrenia research and personalized therapeutic interventions.

Multi-modal Imaging Genomics Transformer: Attentive Integration of Imaging with Genomic Biomarkers for Schizophrenia Classification

Multi-modal deep learning from imaging genomic data for schizophrenia classification

Transcriptomic and neuroimaging data integration enhances machine learning classification of schizophrenia

Exploring Schizophrenia Classification Through Multimodal MRI and Deep Graph Neural Networks: Unveiling Brain Region-Specific Weight Discrepancies and Their Association With Cell-Type Specific Transcriptomic Features

Imaging-genomic Spatial-Modality Attentive Fusion for Studying Neuropsychiatric Disorders.

Integration of Multiple Genomic Imaging Data for the Study of Schizophrenia Using Joint Nonnegative Matrix Factorization.

Multi-SIGATnet: A multimodal schizophrenia MRI classification algorithm using sparse interaction mechanisms and graph attention networks

Classification of First-Episode Schizophrenia Using Multimodal Brain Features: A Combined Structural and Diffusion Imaging Study

Integration of Imaging (epi)genomics Data for the Study of Schizophrenia Using Group Sparse Joint Nonnegative Matrix Factorization

MultiCrossViT: Multimodal Vision Transformer for Schizophrenia Prediction using Structural MRI and Functional Network Connectivity Data

Multi-modal deep learning of functional and structural neuroimaging and genomic data to predict mental illness

Multi-modal Spatial-modality Attentive Fusion for Studying Neuropsychiatric Disorders

An Interpretable Cross-Attentive Multi-modal MRI Fusion Framework for Schizophrenia Diagnosis

Multimodal Analysis of Structural and Functional MRI for Schizophrenia Diagnosis

A multimodal vision transformer for interpretable fusion of functional and structural neuroimaging data

Combining Multi-Modality Data for Searching Biomarkers in Schizophrenia

Deep Multimodal Predictome for Studying Mental Disorders

Multisite Schizophrenia Classification by Integrating Structural Magnetic Resonance Imaging Data with Polygenic Risk Score

Integrating machining learning and multimodal neuroimaging to detect schizophrenia at the level of the individual

MMM: Classification of Schizophrenia Using Multi-Modality Multi-Atlas Feature Representation and Multi-Kernel Learning.

A Framework for Linking Resting-State Chronnectome/genome Features in Schizophrenia: A Pilot Study