Abstract:Understanding the computational mechanisms that underlie the encoding and decoding of environmental stimuli is a paramount investigation within the domain of neuroscience. Central to this pursuit is the exploration of how the brain represents visual information across its hierarchical architecture. A prominent challenge resides in discerning the neural underpinnings of the processing of dynamic natural visual scenes. Although considerable research efforts have been made to characterize individual components of the visual pathway, a systematic understanding of the distinctive neural coding associated with visual stimuli, as they traverse this hierarchical landscape, remains elusive. In this study, we leverage the comprehensive Allen Visual Coding dataset and utilize the capabilities of deep learning neural network models to study the question of neural coding in response to dynamic natural visual scenes across an expansive array of brain regions. We find that our decoding model adeptly deciphers visual scenes from neural spiking patterns exhibited within each distinct brain area. A compelling observation arises from the comparative analysis of decoding performances, which manifests as a notable encoding proficiency within both the visual cortex and subcortical nuclei, in contrast to a relatively diminished encoding activity within hippocampal neurons. Strikingly, our results reveal a robust correlation between our decoding metrics and well-established anatomical and functional hierarchy indexes. These findings not only corroborate existing knowledge in visual coding using artificial visual stimuli but illuminate the functional role of these deeper brain regions using dynamic natural scenes. Consequently, our results proffer a novel perspective on the utility of decoding neural network models as a metric for quantifying the encoding of dynamic natural visual scenes, thereby advancing our comprehension of visual coding within the complex hierarchy of the brain.

Hierarchical dynamic coding coordinates speech comprehension in the brain

The Cortical Maps of Hierarchical Linguistic Structures During Speech Perception

[The Neural Encoding of Continuous Speech - Recent Advances in EEG and MEG Studies].

Cortical Neural Coding of Speech in Simple and Complex Auditory Scenes.

Neural coding of continuous speech in auditory cortex during monaural and dichotic listening.

Time-domain Analysis of Neural Tracking of Hierarchical Linguistic Structures

Evidence of a predictive coding hierarchy in the human brain listening to speech

Elife Assessment: Finding Structure During Incremental Speech Comprehension

Constructing the hierarchy of predictive auditory sequences in the marmoset brain

The Structure and Statistics of Language Jointly Shape Cross-frequency Neural Dynamics During Spoken Language Comprehension

Characterizing Neural Entrainment to Hierarchical Linguistic Units using Electroencephalography (EEG).

Decoding dynamic visual scenes across the brain hierarchy

Cortical tracking of hierarchical linguistic structures in connected speech

A hierarchy of linguistic predictions during natural language comprehension

Tripartite organization of brain state dynamics underlying spoken narrative comprehension

Neural Signatures of Hierarchical Linguistic Structures in Second Language Listening Comprehension

Predictive coding across the left fronto-temporal hierarchy during language comprehension

Information flow across the cortical timescale hierarchy during narrative construction

Neural Dynamics of the Processing of Speech Features: Evidence for a Progression of Features from Acoustic to Sentential Processing

The Temporal Structure of Language Processing in the Human Brain Corresponds to The Layered Hierarchy of Deep Language Models