Abstract:Understanding the computational mechanisms that underlie the encoding and decoding of environmental stimuli is a paramount investigation within the domain of neuroscience. Central to this pursuit is the exploration of how the brain represents visual information across its hierarchical architecture. A prominent challenge resides in discerning the neural underpinnings of the processing of dynamic natural visual scenes. Although considerable research efforts have been made to characterize individual components of the visual pathway, a systematic understanding of the distinctive neural coding associated with visual stimuli, as they traverse this hierarchical landscape, remains elusive. In this study, we leverage the comprehensive Allen Visual Coding dataset and utilize the capabilities of deep learning neural network models to study the question of neural coding in response to dynamic natural visual scenes across an expansive array of brain regions. We find that our decoding model adeptly deciphers visual scenes from neural spiking patterns exhibited within each distinct brain area. A compelling observation arises from the comparative analysis of decoding performances, which manifests as a notable encoding proficiency within both the visual cortex and subcortical nuclei, in contrast to a relatively diminished encoding activity within hippocampal neurons. Strikingly, our results reveal a robust correlation between our decoding metrics and well-established anatomical and functional hierarchy indexes. These findings not only corroborate existing knowledge in visual coding using artificial visual stimuli but illuminate the functional role of these deeper brain regions using dynamic natural scenes. Consequently, our results proffer a novel perspective on the utility of decoding neural network models as a metric for quantifying the encoding of dynamic natural visual scenes, thereby advancing our comprehension of visual coding within the complex hierarchy of the brain.

Sequential sparse autoencoder for dynamic heading representation in ventral intraparietal area

Temporal Spiking Generative Adversarial Networks for Heading Direction Decoding

Simultaneous Spatial-Temporal Decomposition for Connectome-Scale Brain Networks by Deep Sparse Recurrent Auto-Encoder

A Navigation Cognitive System Driven by Hierarchical Spiking Neural Network.

Bayesian Neural Decoding Using A Diversity-Encouraging Latent Representation Learning Method

Decoding region-level visual functions from invasive EEG data

Variational Autoencoder: An Unsupervised Model for Modeling and Decoding fMRI Activity in Visual Cortex

Decoding dynamic visual scenes across the brain hierarchy

Multi-Modal Latent Variables for Cross-Individual Primary Visual Cortex Modeling and Analysis

Spatio-Temporal Encoding of Brain Dynamics with Surface Masked Autoencoders

Visual Guided Dual-spatial Interaction Network for Fine-grained Brain Semantic Decoding

ViF-SD2E: A Robust Weakly-Supervised Method for Neural Decoding

Sparse-Coding Variational Auto-Encoders

Neural Encoding and Decoding with Deep Learning for Dynamic Natural Vision

Decoding Visual Feature Versus Visual Spatial Attention Control with Deep Neural Networks

Predicting proprioceptive cortical anatomy and neural coding with topographic autoencoders

SSVEP-DAN: A Data Alignment Network for SSVEP-based Brain Computer Interfaces

Supervising the Decoder of Variational Autoencoders to Improve Scientific Utility

VAE-Based Generic Decoding Via Subspace Partition and Priori Utilization.

SSVEP-DAN: Cross-Domain Data Alignment for SSVEP-Based Brain–Computer Interfaces

Learning identifiable and interpretable latent models of high-dimensional neural activity using pi-VAE