Abstract:Understanding the computational mechanisms that underlie the encoding and decoding of environmental stimuli is a paramount investigation within the domain of neuroscience. Central to this pursuit is the exploration of how the brain represents visual information across its hierarchical architecture. A prominent challenge resides in discerning the neural underpinnings of the processing of dynamic natural visual scenes. Although considerable research efforts have been made to characterize individual components of the visual pathway, a systematic understanding of the distinctive neural coding associated with visual stimuli, as they traverse this hierarchical landscape, remains elusive. In this study, we leverage the comprehensive Allen Visual Coding dataset and utilize the capabilities of deep learning neural network models to study the question of neural coding in response to dynamic natural visual scenes across an expansive array of brain regions. We find that our decoding model adeptly deciphers visual scenes from neural spiking patterns exhibited within each distinct brain area. A compelling observation arises from the comparative analysis of decoding performances, which manifests as a notable encoding proficiency within both the visual cortex and subcortical nuclei, in contrast to a relatively diminished encoding activity within hippocampal neurons. Strikingly, our results reveal a robust correlation between our decoding metrics and well-established anatomical and functional hierarchy indexes. These findings not only corroborate existing knowledge in visual coding using artificial visual stimuli but illuminate the functional role of these deeper brain regions using dynamic natural scenes. Consequently, our results proffer a novel perspective on the utility of decoding neural network models as a metric for quantifying the encoding of dynamic natural visual scenes, thereby advancing our comprehension of visual coding within the complex hierarchy of the brain.

Testing for context-dependent changes in neural encoding in naturalistic experiments

Low-frequency Cortical Activity Reflects Context-dependent Parsing of Word Sequences

Decoding auditory attention (in real time) with eeg

Tracking Functional Changes in Nonstationary Signals with Evolutionary Ensemble Bayesian Model for Robust Neural Decoding

Exploiting correlations across trials and behavioral sessions to improve neural decoding

Coarse Behavioral Context Decoding

Decoding Neural Activity to Assess Individual Latent State in Ecologically Valid Contexts

Unsupervised decoding of long-term, naturalistic human neural recordings with automated video and audio annotations

Assessing Behavioral and Neural Correlates of Change Detection in Spatialized Acoustic Scenes

Extracting multidimensional stimulus-response correlations using hybrid encoding-decoding of neural activity

Decoding in the fourth dimension: Classification of temporal patterns and their generalization across locations

Encoding and Decoding Models in Cognitive Electrophysiology

Revealing nonlinear neural decoding by analyzing choices

The world seems different in a social context: a neural network analysis of human experimental data

Fractionation of neural reward processing into independent components by novel decoding principle

Decoding Neural Responses in Mouse Visual Cortex through a Deep Neural Network

Bayesian Neural Decoding Using A Diversity-Encouraging Latent Representation Learning Method

Intersubject Dynamic Conditional Correlation: A Novel Method to Track the Framewise Network Implication during Naturalistic Stimuli

Decoding dynamic visual scenes across the brain hierarchy

Context-dependent neural preparation for information relevance vs. probability

A shared neural encoding model for the prediction of subject-specific fMRI response