Modeling of conversational agent with empathy mechanism
Xuyao Dai,Zhen Liu,Tingting Liu,Guokun Zuo,Jialin Xu,Changcheng Shi,Yuanyi Wang
DOI: https://doi.org/10.1016/j.cogsys.2023.101206
IF: 4.541
2024-01-01
Cognitive Systems Research
Abstract:Conversational agents(CA) play a pivotal role in human–computer interaction research. Previous studies have primarily focused on conversation generation, nonverbal behavior expression, and empathy capabilities of CA. However, inconsistencies may arise between the behavior of CA and the context information due to the various relationships present in the multimodal information generated by users. To address this challenge, we have conducted interdisciplinary research aiming to overcome the limitations observed in previous studies regarding empathy mechanism and emotion interactions in CA. In this regard, we have developed a comprehensive framework for multimodal human–computer emotion interaction, enabling CA to recognize human emotions and respond appropriately.The framework comprises a CA with a humanoid embodiment in a virtual reality environment, along with an interactive multimodal emotion recognition-empathetic conversation generation loop architecture. The CA infers user emotions by leveraging multimodal signals, including audio, facial expressions, and conversation text. Subsequently, it exhibits linguistic behavior through an interactive empathetic conversation model. Several experiments related to emotion have been conducted, demonstrating that the CA's multimodal recognition and expression capabilities, along with behavioral consistency, enhance the naturalness and credibility of multimodal human–computer interaction. Overall, this research contributes to the development of a comprehensive framework for multimodal human–computer emotional interaction, enhancing the quality, credibility, and empathy capabilities of CA.
psychology, experimental,neurosciences,computer science, artificial intelligence