Abstract:Background: Schizophrenia (SZ) is a psychiatric condition that adversely affects an individual's cognitive, emotional, and behavioral aspects. The etiology of SZ, although extensively studied, remains unclear, as multiple factors come together to contribute toward its development. There is a consistent body of evidence documenting the presence of structural and functional deviations in the brains of individuals with SZ. Moreover, the hereditary aspect of SZ is supported by the significant involvement of genomics markers. Therefore, the need to investigate SZ from a multi-modal perspective and develop approaches for improved detection arises. Methods: Our proposed method employed a deep learning framework combining features from structural magnetic resonance imaging (sMRI), functional magnetic resonance imaging (fMRI), and genetic markers such as single nucleotide polymorphism (SNP). For sMRI, we used a pre-trained DenseNet to extract the morphological features. To identify the most relevant functional connections in fMRI and SNPs linked to SZ, we applied a 1-dimensional convolutional neural network (CNN) followed by layerwise relevance propagation (LRP). Finally, we concatenated these obtained features across modalities and fed them to the extreme gradient boosting (XGBoost) tree-based classifier to classify SZ from healthy control (HC). Results: Experimental evaluation on clinical dataset demonstrated that, compared to the outcomes obtained from each modality individually, our proposed multi-modal approach performed classification of SZ individuals from HC with an improved accuracy of 79.01%. Conclusion: We proposed a deep learning based framework that selects multi-modal (sMRI, fMRI and genetic) features efficiently and fuse them to obtain improved classification scores. Additionally, by using Explainable AI (XAI), we were able to pinpoint and validate significant functional network connections and SNPs that contributed the most toward SZ classification, providing necessary interpretation behind our findings.

Classification of Schizophrenia Patients and Healthy Controls Using ICA of Complex-Valued Fmri Data and Convolutional Neural Networks

Classification of Schizophrenic Patients and Healthy Controls Using Multiple Spatially Independent Components of Structural MRI Data

Discriminative analysis of schizophrenia patients using an integrated model combining 3D CNN with 2D CNN: A multimodal MR image and connectomics analysis

SSPNet: An interpretable 3D-CNN for classification of schizophrenia using phase maps of resting-state complex-valued fMRI data

A Compound Classification Model for Schizophrenia Based on Brain Fmri and Network Modelling

Classification of First-Episode Schizophrenia Using Multimodal Brain Features: A Combined Structural and Diffusion Imaging Study

Model order effects on ICA of resting-state complex-valued fMRI data: Application to schizophrenia

Classification of EEG-Based Brain Connectivity Networks in Schizophrenia Using a Multi-Domain Connectome Convolutional Neural Network

Automatic schizophrenic discrimination on fNIRS by using complex brain network analysis and SVM

Spatial Source Phase: A New Feature for Identifying Spatial Differences Based on Complex-Valued Resting-State Fmri Data

ICA-based Classification of MCI Vs HC

Automatic Schizophrenia Discrimination on Fnirs by Using PCA and SVM

Multivariate Deep Learning Classification of Alzheimer’s Disease Based on Hierarchical Partner Matching Independent Component Analysis

Classification of Schizophrenia by Seed-based Functional Connectivity using Prefronto-Temporal Functional Near Infrared Spectroscopy

Parallel Group ICA+ICA: Joint Estimation of Linked Functional Network Variability and Structural Covariation with Application to Schizophrenia

Multi-modal deep learning from imaging genomic data for schizophrenia classification

Classification of Schizophrenia Based on Graph Product Depth Neural Network Fusion of Fmri and Dmri Multidimensional Information

Diagnosis of schizophrenia using brain resting-state fMRI with activity maps based on deep learning

Mapping Relationships among Schizophrenia, Bipolar and Schizoaffective Disorders: A Deep Classification and Clustering Framework Using Fmri Time Series

Classification of schizophrenia patients using a graph convolutional network: A combined functional MRI and connectomics analysis

Multimodal classification of drug-naïve first-episode schizophrenia combining anatomical, diffusion and resting state functional resonance imaging.