Abstract:Intra-modal apparent motion has been shown to be affected or 'captured' by information from another, task-irrelevant modality, as shown in cross-modal dynamic capture effect. Here we created inter-modal apparent motion between visual and tactile stimuli and investigated whether there are mutual influences between auditory apparent motion and inter-modal visual/tactile apparent motion. Moreover, we examined whether and how the spatial remapping between somatotopic and external reference frames of tactile events affect the cross-modal capture between auditory apparent motion and inter-modal visual/tactile apparent motion, by introducing two arm postures: arms-uncrossed and arms-crossed. In Experiment 1, we used auditory stimuli (auditory apparent motion) as distractors and inter-modal visual/tactile stimuli (inter-modal apparent motion) as targets while in Experiment 2 we reversed the distractors and targets. In Experiment 1, we found a general detrimental influence of arms-crossed posture in the task of discrimination of direction in visual/tactile stream, but in Experiment 2, the influence of arms-uncrossed posture played a significant role in modulating the inter-modal visual/tactile stimuli capturing over auditory apparent motion. In both Experiments, the synchronously presented motion streams led to noticeable directional congruency effect in judging the target motion. Among the different modality combinations, tactile to tactile apparent motion (TT) and visual to visual apparent motion (VV) are two signatures revealing the asymmetric congruency effects. When the auditory stimuli were targets, the congruency effect was largest with VV distractors, lowest with TT distractors; the pattern was reversed when the auditory stimuli were distractors. In addition, across both experiments the congruency effect in visual to tactile (VT) and tactile to visual (TV) apparent motion was intermediate between the effect-sizes in VV and TT. We replicated the above findings with a block-wise design (Experiment 3). In Experiment 4, we introduced static distractor events (visual or tactile stimulus), and found the modulation of spatial remapping of distractors upon AA motion is reduced. These findings suggest that there are mutual but a robust asymmetric influence between intra-modal auditory apparent motion and intermodal visual/tactile apparent motion. We proposed that relative reliabilities in directional information between distractor and target streams, summed over a remapping process between two spatial reference frames, determined this asymmetric influence.

Capture of auditory motion by vision is represented by an activation shift from auditory to visual motion cortex

Auditory Motion Elicits a Visual Motion Aftereffect

Auditory motion affects visual biological motion processing

Cross-modal integration of auditory and visual motion signals

A Comparison of Visual and Auditory Motion Processing in Human Cerebral Cortex

Audiovisual Synchrony Improves Motion Discrimination via Enhanced Connectivity between Early Visual and Auditory Areas

Apparent motion induces activity suppression in early visual cortex and impairs visual detection.

Congruency effects between auditory and tactile motion: extending the phenomenon of cross-modal dynamic capture

Retinotopic effects during spatial audio-visual integration

Auditory cues facilitate object movement processing in human extrastriate visual cortex during simulated self-motion: A pilot study

In the brain of the beholder: bi-stable motion reveals mesoscopic-scale feedback modulation in V1

Capture of Intermodal Visual/Tactile Apparent Motion by Moving and Static Sound

Mutual influences of intermodal visual/tactile apparent motion and auditory motion with uncrossed and crossed arms.

Combined effects of auditory and visual cues on the perception of vection

Auditory cortex tracks both auditory and visual stimulus dynamics using low-frequency neuronal phase modulation.

Spatial references and audio-tactile interaction in cross-modal dynamic capture.

“Mickey Mousing” in the Brain: Motion-Sound Synesthesia and the Subcortical Substrate of Audio-Visual Integration

Multisensory processing of naturalistic objects in motion: a high-density electrical mapping and source estimation study

Human MT+ mediates perceptual filling-in during apparent motion

Brain representations of motion and position in the double-drift illusion

An Event-Related Potential Study of the Neural Response to Inferred Motion in Visual Images of Varying Coherence.