Abstract:Efficient behavior is supported by humans' ability to rapidly recognize acoustically distinct sounds as members of a common category. Within auditory cortex, there are critical unanswered questions regarding the organization and dynamics of sound categorization. Here, we performed intracerebral recordings in the context of epilepsy surgery as 20 patient-participants listened to natural sounds. We built encoding models to predict neural responses using features of these sounds extracted from different layers within a sound-categorization deep neural network (DNN). This approach yielded highly accurate models of neural responses throughout auditory cortex. The complexity of a cortical site's representation (measured by the depth of the DNN layer that produced the best model) was closely related to its anatomical location, with shallow, middle, and deep layers of the DNN associated with core (primary auditory cortex), lateral belt, and parabelt regions, respectively. Smoothly varying gradients of representational complexity also existed within these regions, with complexity increasing along a posteromedial-to-anterolateral direction in core and lateral belt, and along posterior-to-anterior and dorsal-to-ventral dimensions in parabelt. When we estimated the time window over which each recording site integrates information, we found shorter integration windows in core relative to lateral belt and parabelt. Lastly, we found a relationship between the length of the integration window and the complexity of information processing within core (but not lateral belt or parabelt). These findings suggest hierarchies of timescales and processing complexity, and their interrelationship, represent a functional organizational principle of the auditory stream that underlies our perception of complex, abstract auditory information.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is about the organizational structure and dynamic characteristics of the human auditory cortex when processing natural sounds. Specifically, the researchers want to explore the following key issues: 1. **Sound classification mechanism in the auditory cortex**: How does the auditory cortex organize and process sounds to achieve rapid identification of different sound categories? For example, how can people quickly judge whether a certain sound is a mobile phone vibration or the sound of a bee flying into the room. 2. **Hierarchical structure of sound processing**: Is there a low - level - to - high - level hierarchical structure in the auditory cortex, so that the primary auditory cortex mainly processes acoustic features, while downstream regions encode more abstract representations? The researchers hope to prove the existence of this hierarchical structure through experiments and quantify the representational complexity of different cortical regions. 3. **Relationship between time scale and processing complexity**: Are the time windows for different cortical regions to process sounds different, and how are these time windows related to processing complexity? The researchers hope to reveal the relationship between time scale and processing complexity in order to better understand the functional organization principles of the auditory cortex. To answer these questions, the researchers carried out the following experiments: - **Experimental design**: 20 epilepsy patients listened to a series of natural sounds, including speech, non - speech human vocalizations, music, animal calls and other environmental sounds while undergoing stereoelectroencephalography (sEEG) monitoring. - **Data collection**: The high - gamma activity (HGA) of each patient was recorded, and deep neural networks (DNN) were used to extract sound features and construct encoding models to predict neural responses. - **Data analysis**: By analyzing the neural responses of different cortical regions, the researchers evaluated the representational complexity of each region and explored the relationship between time windows and processing complexity. Finally, the research results show that there is a hierarchical structure in sound processing in the auditory cortex, and the processing complexity and time windows of different regions are different, which provides important insights for understanding the functional organization of the human auditory cortex.

A hierarchy of processing complexity and timescales for natural sounds in human auditory cortex

Temporal coding of speech in human auditory cortex

The Cortical Maps of Hierarchical Linguistic Structures During Speech Perception

Neural representations of complex temporal modulations in the human auditory cortex.

Sound categories are represented as distributed patterns in the human auditory cortex

Brain-optimized extraction of complex sound features that drive continuous auditory perception

Naturalistic Audiovisual Stimulation Reveals the Topographic Organization of Human Auditory Cortex

Sparse representation of neurons for encoding complex sounds in the auditory cortex

Cortical networks representing object categories and high-level attributes of familiar real-world action sounds

Neural dynamics underlying the acquisition of distinct auditory category structures

Cortical Dynamics of Acoustic and Phonological Processing in Speech Perception

Intracranial Mapping of Response Latencies and Task Effects for Spoken Syllable Processing in the Human Brain

Categorical encoding of voice in human superior temporal cortex

Recent advances in understanding the auditory cortex

One hundred ways to process time, frequency, rate and scale in the central auditory system: a pattern-recognition meta-analysis

Responses of auditory-cortex neurons to structural features of natural sounds

Neural responses in human superior temporal cortex support coding of voice representations

A Task-Optimized Neural Network Replicates Human Auditory Behavior, Predicts Brain Responses, and Reveals a Cortical Processing Hierarchy

Neural coding strategies in auditory cortex.

Spectral and Temporal Integration in the Auditory Cortex of Alert Primates

Cortical Oscillations In Auditory Perception And Speech: Evidence For Two Temporal Windows In Human Auditory Cortex