Use of Event-Related Potentials to Assess Visual–Auditory Multisensory Information in the Decision-Making Processes Related to Fast-Consuming Behavior

Zhepeng Rui,Zhenyu Gu
DOI: https://doi.org/10.1080/10447318.2023.2260983
IF: 4.92
2023-01-01
International Journal of Human-Computer Interaction
Abstract:People are accustomed to using mobile phones to purchase products online. Previous studies identified brain activities associated with online buying decisions but did not assess mismatch negativity (MMN), P300 (P3a and P3b), and slow waves. We applied visual animation and sound effects to test visual-auditory multisensory concepts in product promotion strategies. Event-related potential (ERP) responses from 53 participants were analyzed to help design sound effects based on MMN, P300, and slow waves. We designed five short visual-auditory animation videos: visual animation sound effect (VASE), VA sound beep (VASB), visual standard sound beep (VSSB), visual animation-no sound (VANS), and visual standard-no sound (VSNS). These five apps were used to test how matched and unmatched multisensory user experience (UX) influences online shopping decisions. We also used Arrow's impossibility theorem to design the purchasing experiment. VASE evoked MMN (cluster p = 0.0184) compared with VSSB. VASE evoked the strongest P300 (P3a and P3b) responses (cluster p = 1 x 10(-5)), and VASB evoked more negative slow wave amplitudes (cluster p = 1.7 x 10(-4)) compared with VASE. Participants buy products unconsciously via most VASE and VANS mobile apps compared with via VASB and VSNS. Our findings suggest that sensory memory from multisensory UX can promote product consumption in mobile phone shopping interfaces. This study's testing of memory and attention from MMN, P300, and slow waves makes a novel contribution to the marketing discipline. The implications can motivate researchers to apply ERP to analyze the multisensory concept in future interdisciplinary research.
What problem does this paper attempt to address?