Abstract:Within the field of Embodied Conversational Agents (ECAs), the simulation of emotions has been suggested as a means to enhance the believability of ECAs and also to effectively contribute to the goal of more intuitive human–computer interfaces. Although various emotion models have been proposed, results demonstrating the appropriateness of displaying particular emotions within ECA applications are scarce or even inconsistent. Worse, questionnaire methods often seem insufficient to evaluate the impact of emotions expressed by ECAs on users. Therefore we propose to analyze non-conscious physiological feedback (bio-signals) of users within a clearly arranged dynamic interaction scenario where various emotional reactions are likely to be evoked. In addition to its diagnostic purpose, physiological user information is also analyzed online to trigger empathic reactions of the ECA during game play, thus increasing the level of social engagement. To evaluate the appropriateness of different types of affective and empathic feedback, we implemented a cards game called Skip-Bo, where the user plays against an expressive 3D humanoid agent called Max, which was designed at the University of Bielefeld [6] and is based on the emotion simulation system of [2]. Work performed at the University of Tokyo and NII provided a real-time system for empathic (agent) feedback that allows one to derive user emotions from skin conductance and electromyography [13]. The findings of our study indicate that within a competitive gaming scenario, the absence of negative agent emotions is conceived as stress-inducing and irritating, and that the integration of empathic feedback supports the acceptance of Max as a co-equal humanoid opponent.

Bio-Affective Computer Interface for Game Interaction

Automatic Emotional Balancing in Game Design: Use of Emotional Response to Increase Player Immersion

Affective interaction: Using emotions as a user interface in games

Development of emotion recognition interface using complex EEG/ECG bio-signal for interactive contents

Emotion recognition from physiological signals and video games to detect personality traits

Development of Biofeedback Mechanisms in a Procedural Environment Using Biometric Sensors

Evaluating Affective Feedback of the 3D Agent Max in a Competitive Cards Game

EEG-Based Brain-Computer Interface for Emotional Involvement in Games Through Music

BARGAIN: behavioral affective rule-based games adaptation interface–towards emotionally intelligent games: application on a virtual reality environment for socio-moral development

Using subjective emotion, facial expression, and gaze direction to evaluate user affective experience and predict preference when playing single-player games

Emotion Arousal Assessment Based on Multimodal Physiological Signals for Game Users

"AIded with emotions" - a new design approach towards affective computer systems

End-to-end facial and physiological model for Affective Computing and applications

An automated approach to estimate player experience in game events from psychophysiological data

The BIRAFFE2 Experiment. Study in Bio-Reactions and Faces for Emotion-based Personalization for AI Systems

Non-Contact Emotion Recognition Combining Heart Rate and Facial Expression for Interactive Gaming Environments

A personality-guided affective brain—computer interface for implementation of emotional intelligence in machines

An Affective Interaction System Using Virtual Reality and Brain-Computer Interface

Decoding Emotional Experience through Physiological Signal Processing

Multi-modal Affect Detection Using Thermal and Optical Imaging in a Gamified Robotic Exercise