Abstract:Background: Schizophrenia (SZ) is a psychiatric condition that adversely affects an individual's cognitive, emotional, and behavioral aspects. The etiology of SZ, although extensively studied, remains unclear, as multiple factors come together to contribute toward its development. There is a consistent body of evidence documenting the presence of structural and functional deviations in the brains of individuals with SZ. Moreover, the hereditary aspect of SZ is supported by the significant involvement of genomics markers. Therefore, the need to investigate SZ from a multi-modal perspective and develop approaches for improved detection arises. Methods: Our proposed method employed a deep learning framework combining features from structural magnetic resonance imaging (sMRI), functional magnetic resonance imaging (fMRI), and genetic markers such as single nucleotide polymorphism (SNP). For sMRI, we used a pre-trained DenseNet to extract the morphological features. To identify the most relevant functional connections in fMRI and SNPs linked to SZ, we applied a 1-dimensional convolutional neural network (CNN) followed by layerwise relevance propagation (LRP). Finally, we concatenated these obtained features across modalities and fed them to the extreme gradient boosting (XGBoost) tree-based classifier to classify SZ from healthy control (HC). Results: Experimental evaluation on clinical dataset demonstrated that, compared to the outcomes obtained from each modality individually, our proposed multi-modal approach performed classification of SZ individuals from HC with an improved accuracy of 79.01%. Conclusion: We proposed a deep learning based framework that selects multi-modal (sMRI, fMRI and genetic) features efficiently and fuse them to obtain improved classification scores. Additionally, by using Explainable AI (XAI), we were able to pinpoint and validate significant functional network connections and SNPs that contributed the most toward SZ classification, providing necessary interpretation behind our findings.

Classification of Schizophrenia Based on Graph Product Depth Neural Network Fusion of Fmri and Dmri Multidimensional Information

Classification of First-Episode Schizophrenia Using Multimodal Brain Features: A Combined Structural and Diffusion Imaging Study

Exploring Schizophrenia Classification Through Multimodal MRI and Deep Graph Neural Networks: Unveiling Brain Region-Specific Weight Discrepancies and Their Association With Cell-Type Specific Transcriptomic Features

Multi feature fusion network for schizophrenia classification and abnormal brain network recognition

Multi-modal deep learning from imaging genomic data for schizophrenia classification

Classification of schizophrenia patients using a graph convolutional network: A combined functional MRI and connectomics analysis

Mapping Relationships among Schizophrenia, Bipolar and Schizoaffective Disorders: A Deep Classification and Clustering Framework Using Fmri Time Series

Pattern recognition of schizophrenia based on multidimensional spatial feature fusion

Multimodal Analysis of Structural and Functional MRI for Schizophrenia Diagnosis

Multimodal classification of drug-naïve first-episode schizophrenia combining anatomical, diffusion and resting state functional resonance imaging.

Multi-site Diagnostic Classification Of Schizophrenia Using 3D CNN On Aggregated Task-based fMRI Data

Discriminative analysis of schizophrenia patients using an integrated model combining 3D CNN with 2D CNN: A multimodal MR image and connectomics analysis

Diagnosis of schizophrenia using brain resting-state fMRI with activity maps based on deep learning

Multi-SIGATnet: A multimodal schizophrenia MRI classification algorithm using sparse interaction mechanisms and graph attention networks

Multi-Dimension-Embedding-Aware Modality Fusion Transformer for Psychiatric Disorder Clasification

Multi-graph Fusion for Functional Neuroimaging Biomarker Detection

Automatic recognition of schizophrenia from brain-network features using graph convolutional neural network

Discriminative analysis of schizophrenia patients using graph convolutional networks: A combined multimodal MRI and connectomics analysis

MMM: Classification of Schizophrenia Using Multi-Modality Multi-Atlas Feature Representation and Multi-Kernel Learning.

Fmri-Based Brain Disease Diagnosis: A Graph Network Approach

Automatic schizophrenic discrimination on fNIRS by using complex brain network analysis and SVM