Abstract:In recent years, companies have been seeking communication skills from their employees. Increasingly more companies have adopted group discussions during their recruitment process to evaluate the applicants’ communication skills. However, the opportunity to improve communication skills in group discussions is limited because of the lack of partners. To solve this issue as a long-term goal, the aim of this study is to build an autonomous robot that can participate in group discussions, so that its users can repeatedly practice with it. This robot, therefore, has to perform humanlike behaviors with which the users can interact. In this study, the focus was on the generation of two of these behaviors regarding the head of the robot. One is directing its attention to either of the following targets: the other participants or the materials placed on the table. The second is to determine the timings of the robot’s nods. These generation models are considered in three situations: when the robot is speaking, when the robot is listening, and when no participant including the robot is speaking. The research question is: whether these behaviors can be generated end-to-end from and only from the features of peer participants. This work is based on a data corpus containing 2.5 h of the discussion sessions of 10 four-person groups. Multimodal features, including the attention of other participants, voice prosody, head movements, and speech turns extracted from the corpus, were used to train support vector machine models for the generation of the two behaviors. The performances of the generation models of attentional focus were in an F-measure range between 0.4 and 0.6. The nodding model had an accuracy of approximately 0.65. Both experiments were conducted in the setting of leave-one-subject-out cross validation. To measure the perceived naturalness of the generated behaviors, a subject experiment was conducted. In the experiment, the proposed models were compared. They were based on a data-driven method with two baselines: (1) a simple statistical model based on behavior frequency and (2) raw experimental data. The evaluation was based on the observation of video clips, in which one of the subjects was replaced by a robot performing head movements in the above-mentioned three conditions. The experimental results showed that there was no significant difference from original human behaviors in the data corpus and proved the effectiveness of the proposed models.

Nonverbal social behavior generation for social robots using end-to-end learning

Learning Robot Manipulation Skills from Human Demonstration Videos Using Two-Stream 2-D/3-D Residual Networks with Self-Attention

Modeling and evaluating beat gestures for social robots

Forecasting Nonverbal Social Signals during Dyadic Interactions with Generative Adversarial Neural Networks

EMOTION: Expressive Motion Sequence Generation for Humanoid Robots with In-Context Learning

Data-driven emotional body language generation for social robotics

Affect-Driven Learning of Robot Behaviour for Collaborative Human-Robot Interactions

Human-Guided Robot Behavior Learning: A GAN-Assisted Preference-Based Reinforcement Learning Approach

Generation of Head Movements of a Robot Using Multimodal Features of Peer Participants in Group Discussion Conversation

ExGenNet: Learning to Generate Robotic Facial Expression Using Facial Expression Recognition

Generative Expressive Robot Behaviors using Large Language Models

A Data-Efficient Deep Learning Approach for Deployable Multimodal Social Robots

Generating Social-Aware Locations for a Robot in a Human Group by Image Generation Using Adversarial Learning

Why Robots Should Be Social: Enhancing Machine Learning through Social Human-Robot Interaction

Imitating by Generating: Deep Generative Models for Imitation of Interactive Tasks

Speech-Gesture GAN: Gesture Generation for Robots and Embodied Agents

Learning Social Affordance for Human-Robot Interaction

RoboGen: Towards Unleashing Infinite Data for Automated Robot Learning via Generative Simulation

Caption Generation of Robot Behaviors based on Unsupervised Learning of Action Segments

Learning Socially Appropriate Robot Approaching Behavior Toward Groups using Deep Reinforcement Learning

SocialInteractionGAN: Multi-person Interaction Sequence Generation