Abstract:The ventriloquism aftereffect (VAE) refers to a shift in auditory spatial perception following exposure to a spatial disparity between auditory and visual stimuli. The VAE has been previously measured on two distinct time scales. Hundreds or thousands of exposures to a an audio-visual spatial disparity produces enduring VAE that persists after exposure ceases. Exposure to a single audio-visual spatial disparity produces immediate VAE that decays over seconds. To determine if these phenomena are two extremes of a continuum or represent distinct processes, we conducted an experiment with normal hearing listeners that measured VAE in response to a repeated, constant audio-visual disparity sequence, both immediately after exposure to each audio-visual disparity and after the end of the sequence. In each experimental session, subjects were exposed to sequences of auditory and visual targets that were constantly offset by +8° or -8° in azimuth from one another, then localized auditory targets presented in isolation following each sequence. Eye position was controlled throughout the experiment, to avoid the effects of gaze on auditory localization. In contrast to other studies that did not control eye position, we found both a large shift in auditory perception that decayed rapidly after each AV disparity exposure, along with a gradual shift in auditory perception that grew over time and persisted after exposure to the AV disparity ceased. We modeled the temporal and spatial properties of the measured auditory shifts using grey box nonlinear system identification, and found that two models could explain the data equally well. In the power model, the temporal decay of the ventriloquism aftereffect was modeled with a power law relationship. This causes an initial rapid drop in auditory shift, followed by a long tail which accumulates with repeated exposure to audio-visual disparity. In the double exponential model, two separate processes were required to explain the data, one which accumulated and decayed exponentially and the other which slowly integrated over time. Both models fit the data best when the spatial spread of the ventriloquism aftereffect was limited to a window around the location of the audio-visual disparity. We directly compare the predictions made by each model, and suggest additional measurements that could help distinguish which model best describes the mechanisms underlying the VAE.

The effect of amplitude envelope on spatial ventriloquism

Ventriloquism effect with sound stimuli varying in both azimuth and elevation

Both stimulus‐specific and configurational features of multiple visual stimuli shape the spatial ventriloquism effect

Virtually spatialized sounds enhance auditory processing in healthy participants and patients with a disorder of consciousness

Spatially Incongruent Sounds Affect Visual Localization in Virtual Environments.

Multiple time scales of the ventriloquism aftereffect

Auditory Spatial Bisection of Blind and Normally Sighted Individuals in Free Field and Virtual Acoustics

Intersensory binding across space and time: A tutorial review

Perceived multisensory common cause relations shape the ventriloquism effect but only marginally the trial-wise aftereffect

Limits of Perceived Audio-Visual Spatial Coherence as Defined by Reaction Time Measurements

Effects of mismatch negative elicited by auditory deviant of horizontal locations in virtual space

Retinotopic effects during spatial audio-visual integration

Can we hear physical and social space together through prosody?

Dual Representation of the Auditory Space

The impact of spatial variance on precision estimates in an auditory oddball paradigm

Cross-modal correspondence enhances elevation localization in visual-to-auditory sensory substitution

Top-down and bottom-up attention cause the ventriloquism effect with distinct electroencephalography modulations

Accumulation and decay of visual capture and the ventriloquism aftereffect caused by brief audio-visual disparities

Auditory Space Perception Influenced by Visual Spatial Information

Re-weighting of Sound Localization Cues by Audiovisual Training

Identifying spatial effects in a lateralized duration estimation task