Abstract:Decoding of brain tasks aims to identify individuals’ brain states and brain fingerprints to predict behavior. Deep learning provides an important platform for analyzing brain signals at different developmental stages to understand brain dynamics. Due to their internal architecture and feature extraction techniques, existing machine learning and deep-learning approaches for fMRI-based brain decoding must improve classification performance and explainability. The existing approaches also focus on something other than the behavioral traits that can tell about individuals’ variability in behavioral traits. In the current study, we hypothesized that even at the early childhood stage (as early as 3 years), connectivity between brain regions could decode brain tasks and predict behavioural performance in false-belief tasks. To this end, we proposed an explainable deep learning framework to decode brain states (Theory of Mind and Pain states) and predict individual performance on ToM-related false-belief tasks in a developmental dataset. We proposed an explainable spatiotemporal connectivity-based Graph Convolutional Neural Network (Ex-stGCNN) model for decoding brain tasks. Here, we consider a dataset (age range: 3-12 yrs and adults, samples: 155) in which participants were watching a short, soundless animated movie, ”Partly Cloudy,” that activated Theory-of-Mind (ToM) and pain networks. After scanning, the participants underwent a ToMrelated false-belief task, leading to categorization into the pass, fail, and inconsistent groups based on performance. We trained our proposed model using Static Functional Connectivity (SFC) and Inter-Subject Functional Correlations (ISFC) matrices separately. We observed that the stimulus-driven feature set (ISFC) could capture ToM and Pain brain states more accurately with an average accuracy of 94%, whereas it achieved 85% accuracy using SFC matrices. We also validated our results using five-fold cross-validation and achieved an average accuracy of 92%. Besides this study, we applied the SHAP approach to identify neurobiological brain fingerprints that contributed the most to predictions. We hypothesized that ToM network brain connectivity could predict individual performance on false-belief tasks. We proposed an Explainable Convolutional Variational Auto-Encoder model using functional connectivity (FC) to predict individual performance on false-belief tasks and achieved 90% accuracy.

Predicting Task Activation Maps from Resting-State Functional Connectivity using Deep Learning

Prediction of Cognitive Task Activations via Resting-State Functional Connectivity Networks: An EEG Study

From Connectomic to Task-Evoked Fingerprints: Individualized Prediction of Task Contrasts from Resting-State Functional Connectivity

Deep Learning Modeling for Prediction of Cognitive Task Related Features from Resting-state fMRI Data

Beyond functional connectivity: deep learning applied to resting-state fMRI time series in the prediction of 58 human traits in the HCP

A deep learning based approach identifies regions more relevant than resting‐state networks to the prediction of general intelligence from resting‐state fMRI

Brain Age Prediction Based on Resting-State Functional Connectivity Patterns Using Convolutional Neural Networks

Direct comparison of EEG resting state and task functional connectivity patterns for predicting working memory performance using connectome-based predictive modeling

State-space model with deep learning for functional dynamics estimation in resting-state fMRI

Prediction of neurocognition in youth from resting state fMRI

Beyond the Surface: Revealing the Depths of Brain Activity by Predicting fMRI from EEG with Deep Learning

Predicting an individual’s functional connectivity from their structural connectome: Evaluation of evidence, recommendations and future prospects

Rest2Task: Modeling task-specific components in resting-state functional connectivity and applications

Explainable Deep Learning Framework: Decoding Brain Task and Prediction of Individual Performance in False-Belief Task at Early Childhood Stage

A deep learning approach for mental health quality prediction using functional network connectivity and assessment data

DSAM: A Deep Learning Framework for Analyzing Temporal and Spatial Dynamics in Brain Networks

Predicting Alzheimer’s Disease Using Deep Neuro-Functional Networks with Resting-State fMRI

Individual brain activity patterns during task are predicted by distinct resting-state networks that may reflect local neurobiological features

Modeling Motor Task Activation from Resting-State Fmri Using Machine Learning in Individual Subjects

Machine Learning on Dynamic Functional Connectivity: Promise, Pitfalls, and Interpretations

Automated Creativity Prediction Using Natural Language Processing And Resting-State Functional Connectivity: An Fnirs Study