Abstract:Decoding of brain tasks aims to identify individuals’ brain states and brain fingerprints to predict behavior. Deep learning provides an important platform for analyzing brain signals at different developmental stages to understand brain dynamics. Due to their internal architecture and feature extraction techniques, existing machine learning and deep-learning approaches for fMRI-based brain decoding must improve classification performance and explainability. The existing approaches also focus on something other than the behavioral traits that can tell about individuals’ variability in behavioral traits. In the current study, we hypothesized that even at the early childhood stage (as early as 3 years), connectivity between brain regions could decode brain tasks and predict behavioural performance in false-belief tasks. To this end, we proposed an explainable deep learning framework to decode brain states (Theory of Mind and Pain states) and predict individual performance on ToM-related false-belief tasks in a developmental dataset. We proposed an explainable spatiotemporal connectivity-based Graph Convolutional Neural Network (Ex-stGCNN) model for decoding brain tasks. Here, we consider a dataset (age range: 3-12 yrs and adults, samples: 155) in which participants were watching a short, soundless animated movie, ”Partly Cloudy,” that activated Theory-of-Mind (ToM) and pain networks. After scanning, the participants underwent a ToMrelated false-belief task, leading to categorization into the pass, fail, and inconsistent groups based on performance. We trained our proposed model using Static Functional Connectivity (SFC) and Inter-Subject Functional Correlations (ISFC) matrices separately. We observed that the stimulus-driven feature set (ISFC) could capture ToM and Pain brain states more accurately with an average accuracy of 94%, whereas it achieved 85% accuracy using SFC matrices. We also validated our results using five-fold cross-validation and achieved an average accuracy of 92%. Besides this study, we applied the SHAP approach to identify neurobiological brain fingerprints that contributed the most to predictions. We hypothesized that ToM network brain connectivity could predict individual performance on false-belief tasks. We proposed an Explainable Convolutional Variational Auto-Encoder model using functional connectivity (FC) to predict individual performance on false-belief tasks and achieved 90% accuracy.

Comparing interpretation methods in mental state decoding analyses with deep learning models

Challenges for cognitive decoding using deep learning methods

Looking deeper into interpretable deep learning in neuroimaging: a comprehensive survey

Towards best practice of interpreting deep learning models for EEG-based brain computer interfaces

Applications of interpretable deep learning in neuroimaging: a comprehensive review

Relevance Inference Based on Direct Contribution: Counterfactual Explanation to Deep Networks for Intelligent Decision-making

Interpretability of Machine Learning Methods Applied to Neuroimaging

Systematic comparison of 3D Deep learning and classical machine learning explanations for Alzheimer's Disease detection

Interpreting Deep Learning Model Using Rule-based Method

The Cognitive Revolution in Interpretability: From Explaining Behavior to Interpreting Representations and Algorithms

Mental state decoders: game-changers or wishful thinking?

Evaluating deep transfer learning for whole-brain cognitive decoding

Self-Supervised Learning of Brain Dynamics from Broad Neuroimaging Data

Interpreting Deep Learning Models in Natural Language Processing: A Review

Interpretable deep learning: interpretation, interpretability, trustworthiness, and beyond

Explainable Deep Learning Framework: Decoding Brain Task and Prediction of Individual Performance in False-Belief Task at Early Childhood Stage

Deep Learning Decoding of Mental State in Non-invasive Brain Computer Interface

An empirical comparison of different LDA methods in fMRI-based brain states decoding

Interpreting Encoding and Decoding Models

Explaining the black-box model: A survey of local interpretation methods for deep neural networks