Not Perceptually Equivalent in Semantic Emotion Across Visual and Auditory Modalities: Cross-Modal Affective Norms of Two-Character Chinese Emotion-Label Words
Enze Tang,Xinran Fan,Ruomei Fang,Yuhan Zhang,Jie Gong,Jingjing Guan,Hongwei Ding
DOI: https://doi.org/10.1007/s12144-023-05476-2
IF: 2.8
2023-01-01
Current Psychology
Abstract:Over recent years, although multi-sensory studies have increasingly revealed modality-specific mechanisms underlying lexical processing, validated lexical databases with reliable affective norms in both visual and auditory modalities have been scantly established, especially in Chinese. Therefore, this study aims to establish a cross-modal affective database consisting of 350 two-character Chinese emotion-label words, and investigate how neutral speech prosody changes semantic emotion perception in Chinese. Affective ratings of six variables were collected, including familiarity, valence, arousal, dominance, intensity and emotion type, from 364 participants in both visual and auditory modalities. Reliability and validity of the ratings were strictly examined. Statistical analyses manifested the U-shape relationships for valence-arousal and valence-dominance pairwise correlations in the within-modality setting, and identified the existence of a neutral prosody influence on semantic emotion access, thus showing no direct juxtaposition in lexical emotion perception across the two modalities. Specifically, the auditory modality imposed a neutrality convergence on valence perception, decreased the familiarity and dominance feelings, but did not change the intensity parameter. This study is among the first to introduce the multi-modal perspective into Chinese lexical database construction, which not only supplements extant research tools for selecting grammatically homogeneous Chinese emotion-label words as experiment stimuli, but also warrants further investigations on how speech prosody influences lexical semantic perception.