Abstract:Previous work has demonstrated that performance in an auditory selective attention task can be enhanced or impaired, depending on whether a task-irrelevant visual stimulus is temporally coherent with a target auditory stream or with a competing distractor. However, it remains unclear how audiovisual (AV) temporal coherence and auditory selective attention interact at the neurophysiological level. Here, we measured neural activity using EEG while human participants (men and women) performed an auditory selective attention task, detecting deviants in a target audio stream. The amplitude envelope of the two competing auditory streams changed independently, while the radius of a visual disk was manipulated to control the AV coherence. Analysis of the neural responses to the sound envelope demonstrated that auditory responses were enhanced largely independently of the attentional condition: both target and masker stream responses were enhanced when temporally coherent with the visual stimulus. In contrast, attention enhanced the event-related response evoked by the transient deviants, largely independently of AV coherence. These results provide evidence for dissociable neural signatures of bottom-up (coherence) and top-down (attention) effects in AV object formation.SIGNIFICANCE STATEMENT Temporal coherence between auditory stimuli and task-irrelevant visual stimuli can enhance behavioral performance in auditory selective attention tasks. However, how audiovisual temporal coherence and attention interact at the neural level has not been established. Here, we measured EEG during a behavioral task designed to independently manipulate audiovisual coherence and auditory selective attention. While some auditory features (sound envelope) could be coherent with visual stimuli, other features (timbre) were independent of visual stimuli. We find that audiovisual integration can be observed independently of attention for sound envelopes temporally coherent with visual stimuli, while the neural responses to unexpected timbre changes are most strongly modulated by attention. Our results provide evidence for dissociable neural mechanisms of bottom-up (coherence) and top-down (attention) effects on audiovisual object formation.

Spatial organization of EEG coherence during listening to consonant and dissonant chords

Differential Processing of Consonance and Dissonance within the Human Superior Temporal Gyrus

Dissonance encoding in human inferior colliculus covaries with individual differences in dislike of dissonant music

Neural Correlates of Consonance, Dissonance, and the Hierarchy of Musical Pitch in the Human Brainstem

Probing the neural dynamics of musicians' and non-musicians' consonant/dissonant perception: joint analyses of Electrical EncephaloGram (EEG) and Functional Magnetic Resonance Imaging (fMRI)

Early processing of consonance and dissonance in human auditory cortex

Dissonant music engages early visual processing

Musical harmonies and its relationship with emotional processing: An ERP study in young adults

Timbral effects on consonance disentangle psychoacoustic mechanisms and suggest perceptual origins for musical scales

A Study on Coherence of ERP Evoked by Forn and Spatial Location Perception

Hemispheric asymmetries in the processing of temporal acoustic cues in consonant-vowel syllables

The Neural Basis of Tonal Processing in Music: An ALE Meta-Analysis

Neuromagnetic representation of musical roundness in chord progressions

Interactive effects of presentation mode and pitch register on simultaneous consonance

The consonance of chords: The effect of subsemitone changes to each of the notes in a trichord on its perceived consonance

Crossmodal Correspondence Between Tonal Hierarchy and Visual Brightness: Associating Syntactic Structure and Perceptual Dimensions Across Modalities

Dissociable Neural Correlates of Multisensory Coherence and Selective Attention

Neural Correlates of Temporal Complexity and Synchrony During Audiovisual Correspondence Detection

Theta power and coherence illustrate cerebral processing of octave illusion

fMRI Mapping of Brain Activity Associated with the Vocal Production of Consonant and Dissonant Intervals

ERP COHERENCE ANALYSIS OF FORM AND SPATIAL LOCATION PERCEPTION