Abstract:Gestures that accompany speech are an essential part of natural and efficient embodied human communication. The automatic generation of such co‐speech gestures is a long‐standing problem in computer animation and is considered an enabling technology for creating believable characters in film, games, and virtual social spaces, as well as for interaction with social robots. The problem is made challenging by the idiosyncratic and non‐periodic nature of human co‐speech gesture motion, and by the great diversity of communicative functions that gestures encompass. The field of gesture generation has seen surging interest in the last few years, owing to the emergence of more and larger datasets of human gesture motion, combined with strides in deep‐learning‐based generative models that benefit from the growing availability of data. This review article summarizes co‐speech gesture generation research, with a particular focus on deep generative models. First, we articulate the theory describing human gesticulation and how it complements speech. Next, we briefly discuss rule‐based and classical statistical gesture synthesis, before delving into deep learning approaches. We employ the choice of input modalities as an organizing principle, examining systems that generate gestures from audio, text and non‐linguistic input. Concurrent with the exposition of deep learning approaches, we chronicle the evolution of the related training data sets in terms of size, diversity, motion quality, and collection method (e.g., optical motion capture or pose estimation from video). Finally, we identify key research challenges in gesture generation, including data availability and quality; producing human‐like motion; grounding the gesture in the co‐occurring speech in interaction with other speakers, and in the environment; performing gesture evaluation; and integration of gesture synthesis into applications. We highlight recent approaches to tackling the various key challenges, as well as the limitations of these approaches, and point toward areas of future development.

Cultural Self-Adaptive Multimodal Gesture Generation Based on Multiple Culture Gesture Dataset

RGC: Reliable Gesture Classification Via Wearables Using GANs-Based Data Augmentation.

Cross-Culture Multimodal Emotion Recognition With Adversarial Learning

UnifiedGesture: A Unified Gesture Synthesis Model for Multiple Skeletons

DiffuseStyleGesture: Stylized Audio-Driven Co-Speech Gesture Generation with Diffusion Models

Chain of Generation: Multi-Modal Gesture Synthesis via Cascaded Conditional Control

DiM-Gestor: Co-Speech Gesture Generation with Adaptive Layer Normalization Mamba-2

Style Transfer for Co-Speech Gesture Animation: A Multi-Speaker Conditional-Mixture Approach

CoCoGesture: Toward Coherent Co-speech 3D Gesture Generation in the Wild

A Comprehensive Review of Data-Driven Co-Speech Gesture Generation

Editable Co-Speech Gesture Synthesis Enhanced with Individual Representative Gestures

A Large, Crowdsourced Evaluation of Gesture Generation Systems on Common Data: The GENEA Challenge 2020

A Comprehensive Review of Data‐Driven Co‐Speech Gesture Generation

C2G2: Controllable Co-speech Gesture Generation with Latent Diffusion Model

MambaGesture: Enhancing Co-Speech Gesture Generation with Mamba and Disentangled Multi-Modality Fusion

The DiffuseStyleGesture+ entry to the GENEA Challenge 2023

SGED: A Benchmark dataset for Performance Evaluation of Spiking Gesture Emotion Recognition

ConvoFusion: Multi-Modal Conversational Diffusion for Co-Speech Gesture Synthesis

Semantic Gesticulator: Semantics-Aware Co-Speech Gesture Synthesis

EmotionGesture: Audio-Driven Diverse Emotional Co-Speech 3D Gesture Generation

Salient Co-Speech Gesture Synthesizing with Discrete Motion Representation.