Abstract:Understanding the computational mechanisms that underlie the encoding and decoding of environmental stimuli is a paramount investigation within the domain of neuroscience. Central to this pursuit is the exploration of how the brain represents visual information across its hierarchical architecture. A prominent challenge resides in discerning the neural underpinnings of the processing of dynamic natural visual scenes. Although considerable research efforts have been made to characterize individual components of the visual pathway, a systematic understanding of the distinctive neural coding associated with visual stimuli, as they traverse this hierarchical landscape, remains elusive. In this study, we leverage the comprehensive Allen Visual Coding dataset and utilize the capabilities of deep learning neural network models to study the question of neural coding in response to dynamic natural visual scenes across an expansive array of brain regions. We find that our decoding model adeptly deciphers visual scenes from neural spiking patterns exhibited within each distinct brain area. A compelling observation arises from the comparative analysis of decoding performances, which manifests as a notable encoding proficiency within both the visual cortex and subcortical nuclei, in contrast to a relatively diminished encoding activity within hippocampal neurons. Strikingly, our results reveal a robust correlation between our decoding metrics and well-established anatomical and functional hierarchy indexes. These findings not only corroborate existing knowledge in visual coding using artificial visual stimuli but illuminate the functional role of these deeper brain regions using dynamic natural scenes. Consequently, our results proffer a novel perspective on the utility of decoding neural network models as a metric for quantifying the encoding of dynamic natural visual scenes, thereby advancing our comprehension of visual coding within the complex hierarchy of the brain.

Decoding Pixel-Level Image Features From Two-Photon Calcium Signals of Macaque Visual Cortex

Decoding dynamic visual scenes across the brain hierarchy

Large-scale two-photon imaging revealed super-sparse population codes in V1 superficial layer of awake monkeys

Large-scale calcium imaging reveals a systematic V4 map for encoding natural scenes

Nonlinear Decoding of Natural Images From Large-Scale Primate Retinal Ganglion Recordings

Neural Encoding and Decoding with Deep Learning for Dynamic Natural Vision

Robust Decoding of Rich Dynamical Visual Scenes With Retinal Spikes

Large-scale two-photon imaging revealed super-sparse population codes in V1 superficial layer of awake monkeys.

Reconstruction of natural visual scenes from neural spikes with deep neural networks

Complex Pattern Selectivity in Macaque Primary Visual Cortex Revealed by Large-Scale Two-Photon Imaging.

Spiking-timing based pattern recognition with real-world visual stimuli

The brain-inspired decoder for natural visual image reconstruction

Decoding Visual Experience and Mapping Semantics through Whole-Brain Analysis Using fMRI Foundation Models

Neural Decoding of Visual Information Across Different Neural Recording Modalities and Approaches

Large-Scale Two-Photon Maging Revealed Super-Sparse Population Codes In The Vel Superficial Layer Of Awake Monkeys

Classification of A Pressing Movement by Background Removed Single- photon Calcium Image Sequences

Robust Transcoding Sensory Information with Neural Spikes

Decoding Neural Responses in Mouse Visual Cortex through a Deep Neural Network

Unraveling Neural Coding of Dynamic Natural Visual Scenes Via Convolutional Recurrent Neural Networks.

Decoding region-level visual functions from invasive EEG data

A dual‐channel language decoding from brain activity with progressive transfer training