Abstract:Auditory salience is a fundamental property of a sound that allows it to grab a listener's attention regardless of their attentional state or behavioral goals. While previous research has shed light on acoustic factors influencing auditory salience, the semantic dimensions of this phenomenon have remained relatively unexplored owing both to the complexity of measuring salience in audition as well as limited focus on complex natural scenes. In this study, we examine the relationship between acoustic, contextual, and semantic attributes and their impact on the auditory salience of natural audio scenes using a dichotic listening paradigm. The experiments present acoustic scenes in forward and backward directions; the latter allows to diminish semantic effects, providing a counterpoint to the effects observed in forward scenes. The behavioral data collected from a crowd-sourced platform reveal a striking convergence in temporal salience maps for certain sound events, while marked disparities emerge in others. Our main hypothesis posits that differences in the perceptual salience of events are predominantly driven by semantic and contextual cues, particularly evident in those cases displaying substantial disparities between forward and backward presentations. Conversely, events exhibiting a high degree of alignment can largely be attributed to low-level acoustic attributes. To evaluate this hypothesis, we employ analytical techniques that combine rich low-level mappings from acoustic profiles with high-level embeddings extracted from a deep neural network. This integrated approach captures both acoustic and semantic attributes of acoustic scenes along with their temporal trajectories. The results demonstrate that perceptual salience is a careful interplay between low-level and high-level attributes that shapes which moments stand out in a natural soundscape. Furthermore, our findings underscore the important role of longer-term context as a critical component of auditory salience, enabling us to discern and adapt to temporal regularities within an acoustic scene. The experimental and model-based validation of semantic factors of salience paves the way for a complete understanding of auditory salience. Ultimately, the empirical and computational analyses have implications for developing large-scale models for auditory salience and audio analytics.

Semantic richness effects in isolated spoken word recognition: Evidence from massive auditory lexical decision.

Grasping the invisible: Semantic processing of abstract words

Semantic Richness and Aging: The Effect of Number of Features in the Lexical Decision Task

Relative Contributions of Social, Contextual, and Lexical Factors in Speech Processing

Single‐word semantic judgements in semantic dementia: Do phonology and grammatical class count?

Semantic-based Sound Retrieval by ERP in Rapid Serial Auditory Presentation Paradigm.

Memory for Related and Unrelated Words: Further Evidence on the Influence of Semantic Factors in Immediate Serial Recall

Neural semantic effects of tone accents

Semantic feature effect in verbal short-term memory

Influence of social and semantic contexts on phonetic encoding in naturalistic conversations

Shared Features Dominate Semantic Richness Effects for Concrete Concepts

Lexical Diversity, Lexical Sophistication, and Predictability for Speech in Multiple Listening Conditions

A Thousand Words Are Worth a Picture

Are acoustics enough? Semantic effects on auditory salience in natural scenes

The impact of feedback semantics in visual word recognition: number-of-features effects in lexical decision and naming tasks

Number-of-features effects and semantic processing

The role of working memory loads on immediate and long-term sentence recall

The differential roles of lexical and sublexical processing during spoken-word recognition in clear and in noise

The effect of semantic content on the perception of audiovisual movieclips

Sensitivity to orthographic vs. phonological constraints on word recognition: An ERP study with deaf and hearing readers

An Overview of the Neural Circuitry for Sound Processing