Abstract:Auditory salience is a fundamental property of a sound that allows it to grab a listener's attention regardless of their attentional state or behavioral goals. While previous research has shed light on acoustic factors influencing auditory salience, the semantic dimensions of this phenomenon have remained relatively unexplored owing both to the complexity of measuring salience in audition as well as limited focus on complex natural scenes. In this study, we examine the relationship between acoustic, contextual, and semantic attributes and their impact on the auditory salience of natural audio scenes using a dichotic listening paradigm. The experiments present acoustic scenes in forward and backward directions; the latter allows to diminish semantic effects, providing a counterpoint to the effects observed in forward scenes. The behavioral data collected from a crowd-sourced platform reveal a striking convergence in temporal salience maps for certain sound events, while marked disparities emerge in others. Our main hypothesis posits that differences in the perceptual salience of events are predominantly driven by semantic and contextual cues, particularly evident in those cases displaying substantial disparities between forward and backward presentations. Conversely, events exhibiting a high degree of alignment can largely be attributed to low-level acoustic attributes. To evaluate this hypothesis, we employ analytical techniques that combine rich low-level mappings from acoustic profiles with high-level embeddings extracted from a deep neural network. This integrated approach captures both acoustic and semantic attributes of acoustic scenes along with their temporal trajectories. The results demonstrate that perceptual salience is a careful interplay between low-level and high-level attributes that shapes which moments stand out in a natural soundscape. Furthermore, our findings underscore the important role of longer-term context as a critical component of auditory salience, enabling us to discern and adapt to temporal regularities within an acoustic scene. The experimental and model-based validation of semantic factors of salience paves the way for a complete understanding of auditory salience. Ultimately, the empirical and computational analyses have implications for developing large-scale models for auditory salience and audio analytics.

Assessing Behavioral and Neural Correlates of Change Detection in Spatialized Acoustic Scenes

Emergence of Neural Encoding of Auditory Objects While Listening to Competing Speakers

Cortical Neural Coding of Speech in Simple and Complex Auditory Scenes.

Change detection in complex auditory scenes is predicted by auditory memory, pitch perception, and years of musical training

Neural encoding for spatial release from informational masking and its correlation with behavioral metrics

From Beeps to Streets: Unveiling Sensory Input and Relevance Across Auditory Contexts

Auditory change detection and visual selective attention: association between MMN and N2pc

Domain General Change Detection Accounts for "dishabituation" Effects in Temporal-Parietal Regions in Functional Magnetic Resonance Imaging Studies of Speech Perception.

Selective auditory attention within naturalistic scenes modulates reactivity to speech sounds

Attention Modulates the Auditory Cortical Processing of Spatial and Category Cues in Naturalistic Auditory Scenes

Individual Differences in Cognition and Perception Predict Neural Processing of Speech in Noise for Audiometrically Normal Listeners.

Task-dependent neural representations of salient events in dynamic auditory scenes

Are acoustics enough? Semantic effects on auditory salience in natural scenes

Virtually spatialized sounds enhance auditory processing in healthy participants and patients with a disorder of consciousness

Induced Event-Related Coherence Measures during Auditory Change Detection.

Neural Correlates of Temporal Complexity and Synchrony During Audiovisual Correspondence Detection

Examining the lateralization of electrophysiological correlates of auditory awareness

Sound localization in noisy contexts: performance, metacognitive evaluations and head movements

Locating the Sources for Cross-Modal Interactions and Decision Making During Judging the Visual-Affected Auditory Intensity Change.

Bayesian Prior Uncertainty and Surprisal Elicit Distinct Neural Patterns During Sound Localization in Dynamic Environments

Salient Sounds Activate Human Visual Cortex Automatically