Abstract:Brain networks display a hierarchical organization, a complexity that poses a challenge for existing deep learning models, often structured as flat classifiers, leading to difficulties in interpretability and the 'black box' issue. To bridge this gap, we propose a novel architecture: a symbolic autoencoder informed by weak supervision and an Emergent Language (EL) framework. This model moves beyond traditional flat classifiers by producing hierarchical clusters and corresponding imagery, subsequently represented through symbolic sentences to improve the clinical interpretability of hierarchically organized data such as intrinsic brain networks, which can be characterized using resting-state fMRI images. Our innovation includes a generalized hierarchical loss function designed to ensure that both sentences and images accurately reflect the hierarchical structure of functional brain networks. This enables us to model functional brain networks from a broader perspective down to more granular details. Furthermore, we introduce a quantitative method to assess the hierarchical consistency of these symbolic representations. Our qualitative analyses show that our model successfully generates hierarchically organized, clinically interpretable images, a finding supported by our quantitative evaluations. We find that our best performing loss function leads to a hierarchical consistency of over 97% when identifying images corresponding to brain networks. This approach not only advances the interpretability of deep learning models in neuroimaging analysis but also represents a significant step towards modeling the intricate hierarchical nature of brain networks.

What problem does this paper attempt to address?

The paper aims to address the following key issues: ### Research Background and Challenges - **Hierarchical Organization of Brain Networks**: Brain networks exhibit a hierarchical structure, which poses a challenge to existing deep learning models as they are typically flat classifiers, difficult to interpret, and have a "black box" problem. - **Clinical Interpretability**: In clinical applications, understanding the logic behind model predictions is crucial, but deep learning models often lack this interpretability. - **Scarcity of Labeled Data**: Deep learning requires a large amount of labeled data, but such data is usually very limited in the medical field. ### Solution To address the above challenges, the authors propose a novel architecture—**Symbolic Autoencoder**, combining weakly supervised learning and the **Emergent Language (EL) framework**. This model can not only generate hierarchical clustering of brain networks and their corresponding images but also represent this information through symbolic sentences, thereby enhancing clinical interpretability. Specifically: 1. **Hierarchical Loss Function**: A general hierarchical loss function is designed to ensure that the generated sentences and images accurately reflect the hierarchical structure of brain functional networks. 2. **Quantitative Evaluation Method**: A quantitative method is introduced to assess the hierarchical consistency of these symbolic representations. 3. **Model Innovation**: The model can generate images with clear hierarchical organization and clinical interpretability, and represent brain networks from broad to specific perspectives at different levels. ### Main Contributions 1. **Application of Emergent Language to Hierarchical Modeling of Brain Functional Images**: For the first time, emergent language is used for hierarchical analysis of brain functional images, providing a new perspective for brain network research. 2. **Innovative Evaluation Framework**: An evaluation system is established to assess the hierarchical integrity of sentences generated by the emergent language model. 3. **Hierarchically Organized Brain Functional Image Sequences**: Clear and well-organized sequences of brain functional images are generated, demonstrating how the model progressively refines the understanding of network features. ### Experimental Results - **Qualitative Results**: The generated images by the model show a hierarchical progression from parent networks to specific networks, although in some cases, the transition from one symbol to another does not always become more specific but rather shifts. - **Quantitative Results**: The hierarchical quality of sentences generated by the model is evaluated through tests of different hierarchical Independent Component Analysis (ICA) sequences. Results show that at lower ICA orders, the "progressive strict inclusion with bias" loss function performs best; as the ICA order increases, its performance is similar to or slightly worse than other conditions. ### Conclusion This paper introduces an innovative method based on a weakly supervised learning framework. By incorporating a progressive loss function and a new hierarchical consistency evaluation method, it successfully applies emergent language to hierarchical structure medical imaging data, enhancing the interpretability and practicality of the model.

Emergent Language Symbolic Autoencoder (ELSA) with Weak Supervision to Model Hierarchical Brain Networks

A Novel Biologically Inspired ELM-based Network for Image Recognition

NEURO HAND: A weakly supervised Hierarchical Attention Network for interpretable neuroimaging abnormality Detection

Identify Hierarchical Structures from Task-Based fMRI Data via Hybrid Spatiotemporal Neural Architecture Search Net

Sparse Deep Neural Network for Encoding and Decoding the Structural Connectome

Functional2Structural: Cross-Modality Brain Networks Representation Learning

Unsupervised Learning of Neurosymbolic Encoders

A deep learning method for autism spectrum disorder identification based on interactions of hierarchical brain networks

Modeling Task Fmri Data Via Deep Convolutional Autoencoder

Constructing Fine-Granularity Functional Brain Network Atlases Via Deep Convolutional Autoencoder.

Unsupervised representation learning with Hebbian synaptic and structural plasticity in brain-like feedforward neural networks

HDGL: A hierarchical dynamic graph representation learning model for brain disorder classification

The Unsupervised Hierarchical Convolutional Sparse Auto-Encoder for Neuroimaging Data Classification

Deep multivariate autoencoder for capturing complexity in Brain Structure and Behaviour Relationships

Integration of Neural Network-Based Symbolic Regression in Deep Learning for Scientific Discovery

Deep Semantic Architecture with discriminative feature visualization for neuroimage analysis

Deep Variational Lesion-Deficit Mapping

Encoding Multi-Resolution Brain Networks Using Unsupervised Deep Learning

Exploring Brain Effective Connectivity in Visual Perception Using a Hierarchical Correlation Network.

BrainMAE: A Region-aware Self-supervised Learning Framework for Brain Signals

Contrastive Brain Network Learning via Hierarchical Signed Graph Pooling Model