Abstract:Neuropsychiatric disorders such as schizophrenia are very heterogeneous in nature and typically diagnosed using self-reported symptoms. This makes it difficult to pose a confident prediction on the cases and does not provide insight into the underlying neural and biological mechanisms of these disorders. Combining neuroimaging and genomic data with a multi-modal 'predictome' paves the way for biologically informed markers and may improve prediction reliability. With that, we develop a multi-modal deep learning framework by fusing data from different modalities to capture the interaction between the latent features and evaluate their complementary information in characterizing schizophrenia. Our deep model uses structural MRI, functional MRI, and genome-wide polymorphism data to perform the classification task. It includes a multi-layer feed-forward network, an encoder, and a long short-term memory (LSTM) unit with attention to learn the latent features and adopt a joint training scheme capturing synergies between the modalities. The hybrid network also uses different regularizers for addressing the inherent overfitting and modality-specific bias in the multi-modal setup. Next, we run the network through a saliency model to analyze the learned features. Integrating modalities enhances the performance of the classifier, and our framework acquired 88% (P < 0.0001) accuracy on a dataset of 437 subjects. The trimodal accuracy is comparable to the state-of-the-art performance on a data collection of this size and outperforms the unimodal and bimodal baselines we compared. Model introspection was used to expose the salient neural features and genes/biological pathways associated with schizophrenia. To our best knowledge, this is the first approach that fuses genomic information with structural and functional MRI biomarkers for predicting schizophrenia. We believe this type of modality blending can better explain the disorder's dynamics by adding cross-modal prospects.Clinical Relevance- This study combinedly learns imaging and genomic features for the classification of schizophrenia. The data fusion scheme extracts modality interactions, and the saliency experiments report multiple functional and structural networks closely connected to the disorder.

Multi-biological Classification for the Diagnosis of Schizophrenia Using Multi-classifier, Multi-feature Selection and Multi-cross Validation: an Integrated Machine Learning Framework Study

An integrated machine learning framework for a discriminative analysis of schizophrenia using multi-biological data

Multi-modal deep learning from imaging genomic data for schizophrenia classification

Transcriptomic and neuroimaging data integration enhances machine learning classification of schizophrenia

Classification of First-Episode Schizophrenia Using Multimodal Brain Features: A Combined Structural and Diffusion Imaging Study

MMM: Classification of Schizophrenia Using Multi-Modality Multi-Atlas Feature Representation and Multi-Kernel Learning.

Assisting Schizophrenia Diagnosis Using Clinical Electroencephalography and Interpretable Graph Neural Networks: a Real-World and Cross-Site Study

Integrating machining learning and multimodal neuroimaging to detect schizophrenia at the level of the individual

Exploring Schizophrenia Classification Through Multimodal MRI and Deep Graph Neural Networks: Unveiling Brain Region-Specific Weight Discrepancies and Their Association With Cell-Type Specific Transcriptomic Features

Distinguishing Schizophrenia and Bipolar Disorder Through a Multiclass Classification Model Based on Multimodal Neuroimaging Data.

Detection of Schizophrenia Cases From Healthy Controls With Combination of Neurocognitive and Electrophysiological Features

The Effective Diagnosis of Schizophrenia by Using Multi-Layer RBMs Deep Networks

Multisite Schizophrenia Classification by Integrating Structural Magnetic Resonance Imaging Data with Polygenic Risk Score

Mapping Relationships among Schizophrenia, Bipolar and Schizoaffective Disorders: A Deep Classification and Clustering Framework Using Fmri Time Series

Integrating Multi-scale Feature Representation and Ensemble Learning for Schizophrenia Diagnosis.

Effects of Brain Atlases and Machine Learning Methods on the Discrimination of Schizophrenia Patients: A Multimodal MRI Study

Joint multi-site domain adaptation and multi-modality feature selection for the diagnosis of psychiatric disorders

Classification of Schizophrenia by Combination of Brain Effective and Functional Connectivity

Morphometric Integrated Classification Index: A Multisite Model-Based, Interpretable, Shareable and Evolvable Biomarker for Schizophrenia

Multi-modal deep learning of functional and structural neuroimaging and genomic data to predict mental illness

Multimodal classification of drug-naïve first-episode schizophrenia combining anatomical, diffusion and resting state functional resonance imaging.