Abstract:Dynamic emotion is typically used to facilitate human–machine interactions. Conversational data from social media contain a considerable amount of useful information, and such data are the foundation for researching dynamic and artificial emotion. At present, most human–machine interaction systems focus on the complexity and accuracy of the dialog but neglect the emotional characteristics of the speaker. When generating a dialog considering the emotional personality of the interlocutor, controlling, and guiding the dialog to a specified direction are essential. This article presents a system for studying dynamic emotions in human-computer interaction from the perspective of emotional transfer and guidance. Based on the emotional state of the interlocutor and the distribution of emotional transfer, the process of emotional transfer is simulated and sampled, and the sequence of emotional guidance is generated. In this system, two algorithms are proposed. A generative Markov chain Monte Carlo (GEN-MCMC) algorithm is proposed to generate a variety of emotional transfer sequences that fit the talke’s personality dynamically based on the real-world dialog. Further, a guiding MCMC (GUI-MCMC) algorithm-based GEN-MCMC is proposed to generate the emotional guiding sequences. The generated emotional sequences by GEN-MCMC were evaluated in two aspects: 1) consistency and 2) diversity. The experimental results show that the GEN-MCMC algorithm performs better than the general sequence generation algorithm in terms of consistency and diversity in generating emotional states. The GUI-MCMC was able to generate a proper stimulus sequence when given the first and target emotions. An emotional stimulus sequence can simulate the emotional transfer of the interlocutor in the process of dialogue, and give the observer appropriate reference to guide and control the emotions of dialogue. The experimental results show that the proposed system can effectively model the dynamic emotion in emotional transfer and guidance, which can be further used to build chat robots, intelligent assistants, and human–machine interaction systems. The models can also be used for emotional induction and enhance the feel-good or feel-terrible factor in human–machine communication applications, such as medical treatment of mental diseases, interrogation, and psychological attack and defense.

Emotional speech synthesis by XML file using interactive genetic algorithms

Emotional Talking Agent: System and Evaluation

Emotional Audio-Visual Speech Synthesis Based on PAD

Speech Synthesis with Mixed Emotions

A Model of Emotional Speech Generation Based on Conditional Generative Adversarial Networks

EmoEden: Applying Generative Artificial Intelligence to Emotional Learning for Children with High-Function Autism

Prosody Analysis And Modeling For Emotional Speech Synthesis

Emotional speech synthesis with rich and granularized control

Write-a-speaker: Text-based Emotional and Rhythmic Talking-head Generation

Emotional Music Generation Using Interactive Genetic Algorithm

Efficient Emotional Adaptation for Audio-Driven Talking-Head Generation

Toward Synthesizing Expressive Mandarin Speech

EmotiveTalk: Expressive Talking Head Generation through Audio Information Decoupling and Emotional Video Diffusion

Adaptive Algorithm for Feature Selection of Speech Emotion Recognition Based on Genetic Algorithm and SVM

Exemplar-Based Emotive Speech Synthesis

Design and Analysis of a Human–Machine Interaction System for Researching Human’s Dynamic Emotion

Emotion Rendering for Conversational Speech Synthesis with Heterogeneous Graph-Based Context Modeling

Expressive Speech Driven Talking Avatar Synthesis with DBLSTM Using Limited Amount of Emotional Bimodal Data

An Emotional Dialogue System Using Conditional Generative Adversarial Networks with a Sequence-to-Sequence Transformer Encoder

Emotional Human-Machine Conversation Generation Based on Long Short-Term Memory