Abstract:Gestures that accompany speech are an essential part of natural and efficient embodied human communication. The automatic generation of such co‐speech gestures is a long‐standing problem in computer animation and is considered an enabling technology for creating believable characters in film, games, and virtual social spaces, as well as for interaction with social robots. The problem is made challenging by the idiosyncratic and non‐periodic nature of human co‐speech gesture motion, and by the great diversity of communicative functions that gestures encompass. The field of gesture generation has seen surging interest in the last few years, owing to the emergence of more and larger datasets of human gesture motion, combined with strides in deep‐learning‐based generative models that benefit from the growing availability of data. This review article summarizes co‐speech gesture generation research, with a particular focus on deep generative models. First, we articulate the theory describing human gesticulation and how it complements speech. Next, we briefly discuss rule‐based and classical statistical gesture synthesis, before delving into deep learning approaches. We employ the choice of input modalities as an organizing principle, examining systems that generate gestures from audio, text and non‐linguistic input. Concurrent with the exposition of deep learning approaches, we chronicle the evolution of the related training data sets in terms of size, diversity, motion quality, and collection method (e.g., optical motion capture or pose estimation from video). Finally, we identify key research challenges in gesture generation, including data availability and quality; producing human‐like motion; grounding the gesture in the co‐occurring speech in interaction with other speakers, and in the environment; performing gesture evaluation; and integration of gesture synthesis into applications. We highlight recent approaches to tackling the various key challenges, as well as the limitations of these approaches, and point toward areas of future development.

Towards Synchronous Model of Non-emotional Conversational Gesture Generation in Humanoids

Toward Automated Generation of Affective Gestures from Text:A Theory-Driven Approach

Design of conversational humanoid robot based on hardware independent gesture generation

EMOTION: Expressive Motion Sequence Generation for Humanoid Robots with In-Context Learning

Gesture recognition using combination of acceleration sensor and images for casual communication between robots and humans

Biomimetic learning of hand gestures in a humanoid robot

Communicating human intent to a robotic companion by multi-type gesture sentences

Deep Gesture Generation for Social Robots Using Type-Specific Libraries

A Comprehensive Review of Data-Driven Co-Speech Gesture Generation

Speech-Gesture GAN: Gesture Generation for Robots and Embodied Agents

Forecasting Nonverbal Social Signals during Dyadic Interactions with Generative Adversarial Neural Networks

Speech-driven Personalized Gesture Synthetics: Harnessing Automatic Fuzzy Feature Inference

A Framework for Integrating Gesture Generation Models into Interactive Conversational Agents

A Comprehensive Review of Data‐Driven Co‐Speech Gesture Generation

Learning Speech-driven 3D Conversational Gestures from Video

Learning to generate pointing gestures in situated embodied conversational agents

Data-driven emotional body language generation for social robotics

Gesticulator: A framework for semantically-aware speech-driven gesture generation

Speech gesture generation from the trimodal context of text, audio, and speaker identity

Speech2UnifiedExpressions: Synchronous Synthesis of Co-Speech Affective Face and Body Expressions from Affordable Inputs

Moving fast and slow: Analysis of representations and post-processing in speech-driven automatic gesture generation