Abstract:Intra-modal apparent motion has been shown to be affected or 'captured' by information from another, task-irrelevant modality, as shown in cross-modal dynamic capture effect. Here we created inter-modal apparent motion between visual and tactile stimuli and investigated whether there are mutual influences between auditory apparent motion and inter-modal visual/tactile apparent motion. Moreover, we examined whether and how the spatial remapping between somatotopic and external reference frames of tactile events affect the cross-modal capture between auditory apparent motion and inter-modal visual/tactile apparent motion, by introducing two arm postures: arms-uncrossed and arms-crossed. In Experiment 1, we used auditory stimuli (auditory apparent motion) as distractors and inter-modal visual/tactile stimuli (inter-modal apparent motion) as targets while in Experiment 2 we reversed the distractors and targets. In Experiment 1, we found a general detrimental influence of arms-crossed posture in the task of discrimination of direction in visual/tactile stream, but in Experiment 2, the influence of arms-uncrossed posture played a significant role in modulating the inter-modal visual/tactile stimuli capturing over auditory apparent motion. In both Experiments, the synchronously presented motion streams led to noticeable directional congruency effect in judging the target motion. Among the different modality combinations, tactile to tactile apparent motion (TT) and visual to visual apparent motion (VV) are two signatures revealing the asymmetric congruency effects. When the auditory stimuli were targets, the congruency effect was largest with VV distractors, lowest with TT distractors; the pattern was reversed when the auditory stimuli were distractors. In addition, across both experiments the congruency effect in visual to tactile (VT) and tactile to visual (TV) apparent motion was intermediate between the effect-sizes in VV and TT. We replicated the above findings with a block-wise design (Experiment 3). In Experiment 4, we introduced static distractor events (visual or tactile stimulus), and found the modulation of spatial remapping of distractors upon AA motion is reduced. These findings suggest that there are mutual but a robust asymmetric influence between intra-modal auditory apparent motion and intermodal visual/tactile apparent motion. We proposed that relative reliabilities in directional information between distractor and target streams, summed over a remapping process between two spatial reference frames, determined this asymmetric influence.

What You See Depends on What You Hear: Temporal Averaging and Crossmodal Integration

Asymmetrical Cross-Modal Influence on Neural Encoding of Auditory and Visual Features in Natural Scenes.

Temporal Reference, Attentional Modulation, and Crossmodal Assimilation

Auditory Temporal Modulation of the Visual Ternus Effect: the Influence of Time Interval.

Adaptation to Visual or Auditory Time Intervals Modulates the Perception of Visual Apparent Motion

The Speed and Temporal Frequency of Visual Apparent Motion Modulate Auditory Duration Perception

Adaptation to Temporal Interval Modulates the Perception of Visual Apparent Motion

Costs and benefits of audiovisual interactions

Attentional modulations of audiovisual interactions in apparent motion: Temporal ventriloquism effects on perceived visual speed

Selective Adaptation for Temporal Information: Evidence from the Classification of Visual Ternus Apparent Motion

Perception Of Visual Apparent Motion Is Modulated By A Gap Within Concurrent Auditory Glides, Even When It Is Illusory

The Role of Spatiotemporal and Spectral Cues in Segregating Short Sound Events: Evidence from Auditory Ternus Display

Mutual influences of intermodal visual/tactile apparent motion and auditory motion with uncrossed and crossed arms.

Crossmodal Statistical Binding of Temporal Information and Stimuli Properties Recalibrates Perception of Visual Apparent Motion.

Influences of intra- and crossmodal grouping on visual and tactile Ternus apparent motion.

Locating the Sources for Cross-Modal Interactions and Decision Making During Judging the Visual-Affected Auditory Intensity Change.

Temporal ventriloquism along the path of apparent motion: speed perception under different spatial grouping principles

Interaction of Perceptual Grouping and Crossmodal Temporal Capture in Tactile Apparent-Motion

The Development of Audio-Visual Integration for Temporal Judgements

Audiovisual Integration of Natural Auditory and Visual Stimuli in the Real-World Situation

Robust Temporal Averaging of Time Intervals Between Action and Sensation