Abstract:Empathetic response generation, aiming at understanding the user's situation and feelings and respond empathically, is crucial in building human-like dialogue systems. Previous methods mainly focus on using maximum likelihood estimation as the optimization objective for training response generation models, without taking into account the empathy level alignment between generated responses and target responses. To this end, we propose an empathetic response generation using reinforcement learning (EmpRL) framework. The framework designs an effective empathy reward function and generates empathetic responses by maximizing the expected reward through reinforcement learning. Given the powerful text generation capability of pre-trained language models, EmpRL utilizes the pre-trained T5 model as the generator and conducts further training to initialize the policy. To align the empathy level between generated responses and target responses in the context, an empathy reward function containing three empathy communication mechanisms, i.e., emotional reaction, interpretation, and exploration, is constructed using pre-designed and pre-trained empathy identifiers. Finally, the proximal policy optimization algorithm is used to further train the policy to produce empathetic responses. Both automatic and manual evaluations demonstrate that the proposed EmpRL framework can improve the quality of generated responses, enhance the empathy level similarity between generated and target responses, and produce empathetic responses covering both affective and cognitive aspects.

Generating Empathetic Responses by Looking Ahead the User's Sentiment

CEM: Commonsense-aware Empathetic Response Generation

Emotion-Aware Transformer Encoder for Empathetic Dialogue Generation

Empathy Level Alignment via Reinforcement Learning for Empathetic Response Generation

EmpHi: Generating Empathetic Responses with Human-like Intents

Wish I Can Feel What You Feel: A Neural Approach for Empathetic Response Generation

Affective Neural Response Generation

Improving Empathetic Response Generation by Emotion Recognition and Information Filtration

Perspective-taking and Pragmatics for Generating Empathetic Responses Focused on Emotion Causes

Emotional Neural Language Generation Grounded in Situational Contexts

Empathetic Response Generation with State Management

EmpCRL: Controllable Empathetic Response Generation via In-Context Commonsense Reasoning and Reinforcement Learning

Neural Response Generation with Relevant Emotions for Short Text Conversation

Exemplars-guided Empathetic Response Generation Controlled by the Elements of Human Communication

Emotion-aware Chat Machine: Automatic Emotional Response Generation for Human-like Emotional Interaction

Generative Adversarial Imitation Learning for Empathy-based AI

Empathetic Dialogue Generation via Sensitive Emotion Recognition and Sensible Knowledge Selection

Constructing Emotion Consensus and Utilizing Unpaired Data for Empathetic Dialogue Generation

Emotion Analysis for the Upcoming Response in Open-Domain Human-Computer Conversation.

Emotional Human-Machine Conversation Generation Based on Long Short-Term Memory

Neural Conversation Generation with Auxiliary Emotional Supervised Models