The different effects of visual attention and auditory attention on bisensory audiovisual integration

Qi Li,Jingjing Yang,Jinglong Wu

DOI: https://doi.org/10.1109/ICCME.2011.5876760

2011-05-22

CME

Abstract:Two experiments were conducted to explore the different effects of visual/auditory attention on bisensory audiovisual integration using behavioral and electrophysiological measures. Subjects were asked to attend visual stimulus in experiment1, but to attend auditory stimulus in experiment2. In each experiment, unisensory visual, unisensory auditory, and bisensory audiovisual stimuli were presented randomly with equal probability. The subjects were instructed to press a button in response to an infrequent target stimulus (20%) in the attend-designated modality. The behavioral results showed that the responses to audiovisual target stimuli were faster and more accurate than unisensory visual/auditory target stimuli no matter in experiment1 or in experiment2. These findings suggested that when presented simultaneously, visual and auditory stimuli interact to facilitate the behavioral response. The event-related potentials (ERPs) to bisensory stimuli (AV) were compared with the sum of the ERPs to unisensory auditory (A) and visual (V) stimuli. The bisensory audiovisual integration process was expressed as the difference waveform between AV and (A+V). In experiment1, three ERP components related to audiovisual integration were identified: (1) over the right temporal area, around 200–220 ms; (2) over the centro-medial area, around 290–310 ms; and (3) over the left and right ventral temporal area, around 290–310 ms. In experiment2, Two ERP components related to audiovisual integration were identified: (1) over the centro-medial area at 280–300 ms; and (2) over the right fronto-temporal area at 300–320 ms. Our results suggested that attention affects the process of the bisensory audiovisual integration.

Psychology