Abstract:Decoding of cognitive states aims to identify individuals' brain states and brain fingerprints to predict behavior. Deep learning provides an important platform for analyzing brain signals at different developmental stages to understand brain dynamics. Due to their internal architecture and feature extraction techniques, existing machine-learning and deep-learning approaches are suffering from low classification performance and explainability issues that must be improved. In the current study, we hypothesized that even at the early childhood stage (as early as 3-years), connectivity between brain regions could decode brain states and predict behavioral performance in false-belief tasks. To this end, we proposed an explainable deep learning framework to decode brain states (Theory of Mind and Pain states) and predict individual performance on ToM-related false-belief tasks in a developmental dataset. We proposed an explainable spatiotemporal connectivity-based Graph Convolutional Neural Network (Ex-stGCNN) model for decoding brain states. Here, we consider a developmental dataset, N = 155 (122 children; 3-12 yrs and 33 adults; 18-39 yrs), in which participants watched a short, soundless animated movie, shown to activate Theory-of-Mind (ToM) and pain networs. After scanning, the participants underwent a ToM-related false-belief task, leading to categorization into the pass, fail, and inconsistent groups based on performance. We trained our proposed model using Functional Connectivity (FC) and Inter-Subject Functional Correlations (ISFC) matrices separately. We observed that the stimulus-driven feature set (ISFC) could capture ToM and Pain brain states more accurately with an average accuracy of 94%, whereas it achieved 85% accuracy using FC matrices. We also validated our results using five-fold cross-validation and achieved an average accuracy of 92%. Besides this study, we applied the SHapley Additive exPlanations (SHAP) approach to identify brain fingerprints that contributed the most to predictions. We hypothesized that ToM network brain connectivity could predict individual performance on false-belief tasks. We proposed an Explainable Convolutional Variational Auto-Encoder (Ex-Convolutional VAE) model to predict individual performance on false-belief tasks and trained the model using FC and ISFC matrices separately. ISFC matrices again outperformed the FC matrices in prediction of individual performance. We achieved 93.5% accuracy with an F1-score of 0.94 using ISFC matrices and achieved 90% accuracy with an F1-score of 0.91 using FC matrices.

Cognitive Networks and Performance Drive fMRI-Based State Classification Using DNN Models

An Explainable and Generalizable Recurrent Neural Network Approach for Differentiating Human Brain States on EEG Dataset

Real-time Classification of Brain States in Functional MRI Using Dynamic Connectivity Patterns and Machine Learning

Attention module improves both performance and interpretability of four‐dimensional functional magnetic resonance imaging decoding neural network

Differentiating Brain States Via Multi-Clip Random Fragment Strategy-Based Interactive Bidirectional Recurrent Neural Network

Emotional Brain State Classification on fMRI Data Using Deep Residual and Convolutional Networks

Attention module improves both performance and interpretability of 4D fMRI decoding neural network

Decoding the Large-Scale Structure of Brain Function by Classifying Mental States Across Individuals

Interpretable Deep Learning for Neuroimaging-Based Diagnostic Classification

Deep Neural Networks Carve the Brain at its Joints

Taking a Deeper Look at the Brain: Predicting Visual Perceptual and Working Memory Load From High-Density fNIRS Data

Explainable Deep Learning Framework: Decoding Brain Task and Prediction of Individual Performance in False-Belief Task at Early Childhood Stage

Task State Decoding and Mapping of Individual Four-Dimensional Fmri Time Series Using Deep Neural Network

Explainable deep-learning framework: decoding brain states and prediction of individual performance in false-belief task at early childhood stage

Explainable artificial intelligence model to predict brain states from fNIRS signals

Interpretable Cognitive State Prediction Via Temporal Fuzzy Cognitive Map

A Task Performance-guided Model of Functional Networks Identification

Brain Network Classification Based on Dynamic Graph Attention Information Bottleneck.

A Deep Network Model on Dynamic Functional Connectivity with Applications to Gender Classification and Intelligence Prediction

Decoding brain functional connectivity implicated in AD and MCI

Data-driven brain network models predict individual variability in behavior