Abstract:This article provides the first survey of computational models of emotion in reinforcement learning (RL) agents. The survey focuses on agent/robot emotions, and mostly ignores human user emotions. Emotions are recognized as functional in decision-making by influencing motivation and action selection. Therefore, computational emotion models are usually grounded in the agent’s decision making architecture, of which RL is an important subclass. Studying emotions in RL-based agents is useful for three research fields. For machine learning (ML) researchers, emotion models may improve learning efficiency. For the interactive ML and human–robot interaction community, emotions can communicate state and enhance user investment. Lastly, it allows affective modelling researchers to investigate their emotion theories in a successful AI agent class. This survey provides background on emotion theory and RL. It systematically addresses (1) from what underlying dimensions (e.g. homeostasis, appraisal) emotions can be derived and how these can be modelled in RL-agents, (2) what types of emotions have been derived from these dimensions, and (3) how these emotions may either influence the learning efficiency of the agent or be useful as social signals. We also systematically compare evaluation criteria, and draw connections to important RL sub-domains like (intrinsic) motivation and model-based RL. In short, this survey provides both a practical overview for engineers wanting to implement emotions in their RL agents, and identifies challenges and directions for future emotion-RL research.

Survey and perspective on social emotions in robotics

Survey and perspective on social emotions in robotics

A Survey of Robotics and Emotion: Classifications and Models of Emotional Interaction

Emotion Space Modelling for Social Robots

Modeling emotion for human-like behavior in future intelligent robots

A Survey on Emotion Recognition for Human Robot Interaction

Perception of Emotions in Human and Robot Faces: Is the Eye Region Enough?

A Mini-survey on Psychological Pillars of Empathy for Social Robots: Self-Awareness, Theory of Mind, and Perspective Taking

Past, Present, and Future: A Survey of The Evolution of Affective Robotics For Well-being

Emotion Recognition for Human-Robot Interaction: Recent Advances and Future Perspectives

Survey of Design Paradigms for Social Robots

Is It Me or the Robot? A Critical Evaluation of Human Affective State Recognition in a Cognitive Task

Robots’ “Woohoo” and “Argh” can Enhance Users’ Emotional and Social Perceptions: An Exploratory Study on Non-Lexical Vocalizations and Non-Linguistic Sounds

Analysis of Influencing Factors on Humanoid Robots' Emotion Expressions by Body Language.

Can my robotic home cleaner be happy? Issues about emotional expression in non-bio-inspired robots

Artificial Emotions: Toward a Human-Centric Ethics

Empathy in Human–Robot Interaction: Designing for Social Robots

Facial emotion expressions in human-robot interaction: A survey

Recognizing Emotional Body Language Displayed by a Human-like Social Robot

Emotion in reinforcement learning agents and robots: a survey

A MultiModal Social Robot Toward Personalized Emotion Interaction