Abstract:In recent years, companies have been seeking communication skills from their employees. Increasingly more companies have adopted group discussions during their recruitment process to evaluate the applicants’ communication skills. However, the opportunity to improve communication skills in group discussions is limited because of the lack of partners. To solve this issue as a long-term goal, the aim of this study is to build an autonomous robot that can participate in group discussions, so that its users can repeatedly practice with it. This robot, therefore, has to perform humanlike behaviors with which the users can interact. In this study, the focus was on the generation of two of these behaviors regarding the head of the robot. One is directing its attention to either of the following targets: the other participants or the materials placed on the table. The second is to determine the timings of the robot’s nods. These generation models are considered in three situations: when the robot is speaking, when the robot is listening, and when no participant including the robot is speaking. The research question is: whether these behaviors can be generated end-to-end from and only from the features of peer participants. This work is based on a data corpus containing 2.5 h of the discussion sessions of 10 four-person groups. Multimodal features, including the attention of other participants, voice prosody, head movements, and speech turns extracted from the corpus, were used to train support vector machine models for the generation of the two behaviors. The performances of the generation models of attentional focus were in an F-measure range between 0.4 and 0.6. The nodding model had an accuracy of approximately 0.65. Both experiments were conducted in the setting of leave-one-subject-out cross validation. To measure the perceived naturalness of the generated behaviors, a subject experiment was conducted. In the experiment, the proposed models were compared. They were based on a data-driven method with two baselines: (1) a simple statistical model based on behavior frequency and (2) raw experimental data. The evaluation was based on the observation of video clips, in which one of the subjects was replaced by a robot performing head movements in the above-mentioned three conditions. The experimental results showed that there was no significant difference from original human behaviors in the data corpus and proved the effectiveness of the proposed models.

Human Behavior Modeling During Dialogue by Using Generative Adversarial Networks

Adversarial Learning for Neural Dialogue Generation.

A Generative Model to Embed Human Expressivity into Robot Motions

Generative Modeling of Multimodal Multi-Human Behavior

Generation of Head Movements of a Robot Using Multimodal Features of Peer Participants in Group Discussion Conversation

Human Action Generation with Generative Adversarial Networks

Interactive Generative Adversarial Networks for Facial Expression Generation in Dyadic Interactions

Speech-Gesture GAN: Gesture Generation for Robots and Embodied Agents

Modeling and evaluating beat gestures for social robots

MotionChain: Conversational Motion Controllers via Multimodal Prompts

Learning a Generative Model for Multi‐Step Human‐Object Interactions from Videos

Mani-GPT: A Generative Model for Interactive Robotic Manipulation

Learning to gesticulate by observation using a deep generative approach

Towards Synchronous Model of Non-emotional Conversational Gesture Generation in Humanoids

Forecasting Nonverbal Social Signals during Dyadic Interactions with Generative Adversarial Neural Networks

Generative Expressive Robot Behaviors using Large Language Models

Dyadic Interaction Modeling for Social Behavior Generation

Generative Modeling Perspective for Control and Reasoning in Robotics

Imitation of human motion achieves natural head movements for humanoid robots in an active-speaker detection task

Harmon: Whole-Body Motion Generation of Humanoid Robots from Language Descriptions

Motion-Agent: A Conversational Framework for Human Motion Generation with LLMs