Abstract:The ventriloquism aftereffect (VAE) refers to a shift in auditory spatial perception following exposure to a spatial disparity between auditory and visual stimuli. The VAE has been previously measured on two distinct time scales. Hundreds or thousands of exposures to a an audio-visual spatial disparity produces enduring VAE that persists after exposure ceases. Exposure to a single audio-visual spatial disparity produces immediate VAE that decays over seconds. To determine if these phenomena are two extremes of a continuum or represent distinct processes, we conducted an experiment with normal hearing listeners that measured VAE in response to a repeated, constant audio-visual disparity sequence, both immediately after exposure to each audio-visual disparity and after the end of the sequence. In each experimental session, subjects were exposed to sequences of auditory and visual targets that were constantly offset by +8° or -8° in azimuth from one another, then localized auditory targets presented in isolation following each sequence. Eye position was controlled throughout the experiment, to avoid the effects of gaze on auditory localization. In contrast to other studies that did not control eye position, we found both a large shift in auditory perception that decayed rapidly after each AV disparity exposure, along with a gradual shift in auditory perception that grew over time and persisted after exposure to the AV disparity ceased. We modeled the temporal and spatial properties of the measured auditory shifts using grey box nonlinear system identification, and found that two models could explain the data equally well. In the power model, the temporal decay of the ventriloquism aftereffect was modeled with a power law relationship. This causes an initial rapid drop in auditory shift, followed by a long tail which accumulates with repeated exposure to audio-visual disparity. In the double exponential model, two separate processes were required to explain the data, one which accumulated and decayed exponentially and the other which slowly integrated over time. Both models fit the data best when the spatial spread of the ventriloquism aftereffect was limited to a window around the location of the audio-visual disparity. We directly compare the predictions made by each model, and suggest additional measurements that could help distinguish which model best describes the mechanisms underlying the VAE.

An event-related potential investigation of the time-course of temporal ventriloquism

Grouping by feature of cross-modal flankers in temporal ventriloquism

The Ventriloquist Illusion as a Tool to Study Multisensory Processing: An Update

Splitting time: Sound-induced illusory visual temporal fission and fusion.

Limits of Perceived Audio-Visual Spatial Coherence as Defined by Reaction Time Measurements

Intersensory binding across space and time: A tutorial review

Multiple time scales of the ventriloquism aftereffect

Visual modulation of auditory evoked potentials in the cat

Both stimulus‐specific and configurational features of multiple visual stimuli shape the spatial ventriloquism effect

Ventriloquism effect with sound stimuli varying in both azimuth and elevation

Top-down and bottom-up attention cause the ventriloquism effect with distinct electroencephalography modulations

Cortical Oscillations In Auditory Perception And Speech: Evidence For Two Temporal Windows In Human Auditory Cortex

Violation of the Unity Assumption Disrupts Temporal Ventriloquism Effect in Starlings

Anticipatory and evoked visual cortical dynamics of voluntary temporal attention

Using event-related brain potentials to evaluate motor-auditory latencies in virtual reality

Attentional modulations of audiovisual interactions in apparent motion: Temporal ventriloquism effects on perceived visual speed

The effect of amplitude envelope on spatial ventriloquism

Startling acoustic stimuli hasten reflexive choice reaching tasks by strengthening, but not changing the timing of, express visuomotor responses

Time-varying networks of ERPs in P300-speller paradigms based on spatially and semantically congruent audiovisual bimodality

Contextual Neural Dynamics During Time Perception in Primate Ventral Premotor Cortex