CrossModal Correspondence based MultisensoryIntegration: A pilot study showing how HAV cues can modulate the reaction time

Swati Banerjee,Daria Shumkova
DOI: https://doi.org/10.1101/2024.03.21.586134
2024-03-27
Abstract:We live in a multisensory world, where all our senses work together for giving us a fulfilling experience of the environment that we are in or during our use of immersive technologies. For gaining more insight into the temporal scale understanding of the integration phenomenon EEG based BCI can give us the understanding of the transient changes in the brain. In this study, we investigated the potential of incorporating haptics into crossmodal correspondence based research to induce MSI effect through either the active touch users’ feedback or crossmodal correspondences with visual and auditory modalities, such as Kiki Bouba effect. We designed two experiments: Results suggested that it is indeed possible to achieve this type of integration without relying on complex haptic devices. Introducing haptics into BCI technologies through feedback touch or crossmodal correspondances holds potential to improve the user experience and information transfer rate (ITR). Participants, as expected, showed the lowest reaction times in congruent sequential test and the highest – in incongruent HAV cues based test. This indicates the importance preference for sequential cue presentation over simultaneous one. The time was significantly higher in case of Incongruent Haptic cues.
Animal Behavior and Cognition
What problem does this paper attempt to address?