Abstract:Decoding of cognitive states aims to identify individuals' brain states and brain fingerprints to predict behavior. Deep learning provides an important platform for analyzing brain signals at different developmental stages to understand brain dynamics. Due to their internal architecture and feature extraction techniques, existing machine-learning and deep-learning approaches are suffering from low classification performance and explainability issues that must be improved. In the current study, we hypothesized that even at the early childhood stage (as early as 3-years), connectivity between brain regions could decode brain states and predict behavioral performance in false-belief tasks. To this end, we proposed an explainable deep learning framework to decode brain states (Theory of Mind and Pain states) and predict individual performance on ToM-related false-belief tasks in a developmental dataset. We proposed an explainable spatiotemporal connectivity-based Graph Convolutional Neural Network (Ex-stGCNN) model for decoding brain states. Here, we consider a developmental dataset, N = 155 (122 children; 3-12 yrs and 33 adults; 18-39 yrs), in which participants watched a short, soundless animated movie, shown to activate Theory-of-Mind (ToM) and pain networs. After scanning, the participants underwent a ToM-related false-belief task, leading to categorization into the pass, fail, and inconsistent groups based on performance. We trained our proposed model using Functional Connectivity (FC) and Inter-Subject Functional Correlations (ISFC) matrices separately. We observed that the stimulus-driven feature set (ISFC) could capture ToM and Pain brain states more accurately with an average accuracy of 94%, whereas it achieved 85% accuracy using FC matrices. We also validated our results using five-fold cross-validation and achieved an average accuracy of 92%. Besides this study, we applied the SHapley Additive exPlanations (SHAP) approach to identify brain fingerprints that contributed the most to predictions. We hypothesized that ToM network brain connectivity could predict individual performance on false-belief tasks. We proposed an Explainable Convolutional Variational Auto-Encoder (Ex-Convolutional VAE) model to predict individual performance on false-belief tasks and trained the model using FC and ISFC matrices separately. ISFC matrices again outperformed the FC matrices in prediction of individual performance. We achieved 93.5% accuracy with an F1-score of 0.94 using ISFC matrices and achieved 90% accuracy with an F1-score of 0.91 using FC matrices.

<i>Neuro</i>Explainer: Fine-Grained Attention Decoding to Uncover Cortical Development Patterns of Preterm Infants

UniVisNet: A Unified Visualization and Classification Network for accurate of from MRI

Deep Learning with Explainability for Characterizing Age-Related Intrinsic Differences in Dynamic Brain Functional Connectivity.

Less is More: Discovering Concise Network Explanations

An Explainable Deep Learning Framework for Characterizing and Interpreting Human Brain States.

A Novel Explainable Neural Network for Alzheimer’s Disease Diagnosis

BrainNNExplainer: An Interpretable Graph Neural Network Framework for Brain Network based Disease Analysis

Automated neonatal nnU-Net brain MRI extractor trained on a large multi-institutional dataset

An Attention-Based Hemispheric Relation Inference Network for Perinatal Brain Age Prediction

An Explainable 3D Residual Self-Attention Deep Neural Network FOR Joint Atrophy Localization and Alzheimer's Disease Diagnosis using Structural MRI

Explainable deep-learning framework: decoding brain states and prediction of individual performance in false-belief task at early childhood stage

Solving the enigma: Deriving optimal explanations of deep networks

Hierarchical Brain Embedding Using Explainable Graph Learning.

Never‐Ending Learning for Explainable Brain Computing

A Multi-Task, Multi-Stage Deep Transfer Learning Model for Early Prediction of Neurodevelopment in Very Preterm Infants

Explainable Deep Learning Framework: Decoding Brain Task and Prediction of Individual Performance in False-Belief Task at Early Childhood Stage

Differentiate Preterm and Term Infant Brains and Characterize the Corresponding Biomarkers Via DICCCOL-based Multi-Modality Graph Neural Networks

Attention module improves both performance and interpretability of 4D fMRI decoding neural network

A Deep Network for Explainable Prediction of Non-Imaging Phenotypes using Anatomical Multi-View Data

Learning local discrete features in explainable-by-design convolutional neural networks

Attention module improves both performance and interpretability of four‐dimensional functional magnetic resonance imaging decoding neural network