Abstract:Background: The integration of the information encoded in multiparametric MRI images can enhance the performance of machine-learning classifiers. In this study, we investigate whether the combination of structural and functional MRI might improve the performances of a deep learning (DL) model trained to discriminate subjects with Autism Spectrum Disorders (ASD) with respect to typically developing controls (TD). Material and methods: We analyzed both structural and functional MRI brain scans publicly available within the ABIDE I and II data collections. We considered 1383 male subjects with age between 5 and 40 years, including 680 subjects with ASD and 703 TD from 35 different acquisition sites. We extracted morphometric and functional brain features from MRI scans with the Freesurfer and the CPAC analysis packages, respectively. Then, due to the multisite nature of the dataset, we implemented a data harmonization protocol. The ASD vs. TD classification was carried out with a multiple-input DL model, consisting in a neural network which generates a fixed-length feature representation of the data of each modality (FR-NN), and a Dense Neural Network for classification (C-NN). Specifically, we implemented a joint fusion approach to multiple source data integration. The main advantage of the latter is that the loss is propagated back to the FR-NN during the training, thus creating informative feature representations for each data modality. Then, a C-NN, with a number of layers and neurons per layer to be optimized during the model training, performs the ASD-TD discrimination. The performance was evaluated by computing the Area under the Receiver Operating Characteristic curve within a nested 10-fold cross-validation. The brain features that drive the DL classification were identified by the SHAP explainability framework. Results: The AUC values of 0.66±0.05 and of 0.76±0.04 were obtained in the ASD vs. TD discrimination when only structural or functional features are considered, respectively. The joint fusion approach led to an AUC of 0.78±0.04. The set of structural and functional connectivity features identified as the most important for the two-class discrimination supports the idea that brain changes tend to occur in individuals with ASD in regions belonging to the Default Mode Network and to the Social Brain. Conclusions: Our results demonstrate that the multimodal joint fusion approach outperforms the classification results obtained with data acquired by a single MRI modality as it efficiently exploits the complementarity of structural and functional brain information.

MAFT-SO: A novel multi-atlas fusion template based on spatial overlap for ASD diagnosis

MAGE: Automatic diagnosis of autism spectrum disorders using multi-atlas graph convolutional networks and ensemble learning

AIMAFE: Autism Spectrum Disorder Identification with Multi-Atlas Deep Feature Representation and Ensemble Learning

A Multiview Brain Network Transformer Fusing Individualized Information for Autism Spectrum Disorder Diagnosis

A Multi-Graph Cross-Attention based Region-Aware Feature Fusion Network using Multi-Template for Brain Disorder Diagnosis

Multi-atlas Representations Based on Graph Convolutional Networks for Autism Spectrum Disorder Diagnosis.

MADE-for-ASD: A Multi-Atlas Deep Ensemble Network for Diagnosing Autism Spectrum Disorder

A multi-view convolutional neural network method combining attention mechanism for diagnosing autism spectrum disorder

Deep learning based joint fusion approach to exploit anatomical and functional brain information in autism spectrum disorders

Enhancing the Representation of Functional Connectivity Networks by Fusing Multi-View Information for Autism Spectrum Disorder Diagnosis

Self-weighted Adaptive Structure Learning for ASD Diagnosis Via Multi-Template Multi-Center Representation

Fusion of Fmri Multisource Data Using a Two-stream Separable 3D Network for the Diagnosis of Autism Spectrum Disorder

Research on multi‐center assisted diagnosis of ASD based on multimodal feature fusion

Deep Fusion of Multi-Template using Spatio-Temporal Weighted Multi-Hypergraph Convolutional Networks for Brain Disease Analysis

Multi-scale Convolutional Network for fMRI-Based Diagnosis of Autism Spectrum Disorder

ST-ASDNET: A BLSTM-FCN-Transformer basedASD Classification Model for Time-series fMRI

Multi-Class ASD Classification Based on Functional Connectivity and Functional Correlation Tensor via Multi-Source Domain Adaptation and Multi-View Sparse Representation

Interpretable Feature Learning Using Multi-output Takagi-Sugeno-Kang Fuzzy System for Multi-center ASD Diagnosis.

Multi-Atlas Brain Network Classification through Consistency Distillation and Complementary Information Fusion

Integrating Genomic and Resting State Fmri for Efficient Autism Spectrum Disorder Classification

Multisite Autism Spectrum Disorder Classification Using Convolutional Neural Network Classifier and Individual Morphological Brain Networks