Abstract:The use of expressive Virtual Characters is an effective complementary means of communication for social networks offering multi-user 3D-chatting environment. In such contexts, the facial expression channel offers a rich medium to translate the ongoing emotions conveyed by the text-based exchanges. However, until recently, only purely symmetric facial expressions have been considered for that purpose. In this article we examine human sensitivity to facial asymmetry in the expression of both basic and complex emotions. The rationale for introducing asymmetry in the display of facial expressions stems from two well-established observations in cognitive neuroscience: first that the expression of basic emotions generally displays a small asymmetry, second that more complex emotions such as ambivalent feeling may reflect in the partial display of different, potentially opposite, emotions on each side of the face. A frequent occurrence of this second case results from the conflict between the truly felt emotion and the one that should be displayed due to social conventions. Our main hypothesis is that a much larger expressive and emotional space can only be automatically synthesized by means of facial asymmetry when modeling emotions with a general Valence-Arousal-Dominance dimensional approach. Besides, we want also to explore the general human sensitivity to the introduction of a small degree of asymmetry into the expression of basic emotions. We conducted an experiment by presenting 64 pairs of static facial expressions, one symmetric and one asymmetric, illustrating eight emotions (three basic and five complex ones) alternatively for a male and a female character. Each emotion was presented four times by swapping the symmetric and asymmetric positions and by mirroring the asymmetrical expression. Participants were asked to grade, on a continuous scale, the correctness of each facial expression with respect to a short definition. Results confirm the potential of introducing facial asymmetry for a subset of the complex emotions. Guidelines are proposed for designers of embodied conversational agent and emotionally reflective avatars.

Analysis of emotional effect on speech-body gesture interplay.

Decoding spatiotemporal features of emotional body language in social interactions

Beyond Facial Expressions: Learning Human Emotion from Body Gestures.

Emotional Head Motion Predicting from Prosodic and Linguistic Features

Spatiotemporal structure and substates in facial expressions and speech movements

Evaluating the Sensitivity to Virtual Characters Facial Asymmetry in Emotion Synthesis

Enhancing Emotional Induction in Virtual Reality: Innovative Fusion of Speech and Realistic Non-Verbal Emotional Expression

Investigation of the Relation Between Acoustic Features and Articulation — an Application to Emotional Speech Analysis

Acoustic and Physiological Feature Analysis of Affective Speech

EmotionGesture: Audio-Driven Diverse Emotional Co-Speech 3D Gesture Generation

Emotional Talking Agent: System and Evaluation

Concordance Between Facial Micro-Expressions and Physiological Signals under Emotion Elicitation

A Novel Emotion-Aware Method Based on the Fusion of Textual Description of Speech, Body Movements, and Facial Expressions

AffectEcho: Speaker Independent and Language-Agnostic Emotion and Affect Transfer for Speech Synthesis

Body sway reflects joint emotional expression in music ensemble performance

EffMulti: Efficiently Modeling Complex Multimodal Interactions for Emotion Analysis

The expression of emotions by text and speech

Happy Companion : A System of Multimodal Human-Computer Affective Interaction

EMoG: Synthesizing Emotive Co-speech 3D Gesture with Diffusion Model

A generic framework for editing and synthesizing multimodal data with relative emotion strength