Abstract:Transferring the motion style from one animation clip to another, while preserving the motion content of the latter, has been a long-standing problem in character animation. Most existing data-driven approaches are supervised and rely on paired data, where motions with the same content are performed in different styles. In addition, these approaches are limited to transfer of styles that were seen during training. In this paper, we present a novel data-driven framework for motion style transfer, which learns from an unpaired collection of motions with style labels, and enables transferring motion styles not observed during training. Furthermore, our framework is able to extract motion styles directly from videos, bypassing 3D reconstruction, and apply them to the 3D input motion. Our style transfer network encodes motions into two latent codes, for content and for style, each of which plays a different role in the decoding (synthesis) process. While the content code is decoded into the output motion by several temporal convolutional layers, the style code modifies deep features via temporally invariant adaptive instance normalization (AdaIN). Moreover, while the content code is encoded from 3D joint rotations, we learn a common embedding for style from either 3D or 2D joint positions, enabling style extraction from videos. Our results are comparable to the state-of-the-art, despite not requiring paired training data, and outperform other methods when transferring previously unseen styles. To our knowledge, we are the first to demonstrate style transfer directly from videos to 3D animations - an ability which enables one to extend the set of style examples far beyond motions captured by MoCap systems.

Style Synthesis and Editing of Motion Data in Non-Linear Subspace

3D Human Motion Synthesis Based on Nonlinear Manifold Learning

Motion Synthesis Using Style-Editable Inverse Kinematics

Supervised Learning of Motion Style for Real-time Synthesis of 3D Character Animations

A Stylistic Human Motion Editing System Based on A Subspace Motion Model

Diverse Image Style Transfer Via Invertible Cross-Space Mapping

Key-styling: learning motion style for real-time synthesis of 3D animation

Correlation-based and Content-Enhanced Network for Video Style Transfer

Style Subspaces for Character Animation

Key-styling: learning motion style for real-time synthesis of 3D animation: Research Articles

Learning Style-Directed Dynamics Of Human Motion For Automatic Motion Synthesis

On-Line motion style transfer

Real-Time Style Modelling of Human Locomotion via Feature-Wise Transformations and Local Motion Phases

Diffusion‐based Human Motion Style Transfer with Semantic Guidance

Unpaired motion style transfer from video to animation

Diffusion-based Human Motion Style Transfer with Semantic Guidance

FineStyle: Semantic-Aware Fine-Grained Motion Style Transfer with Dual Interactive-Flow Fusion

Algorithm for Real-time Style Transfer for Human Motion

Real-Time Synthesis of 3d Animations by Learning Parametric Gaussians Using Self-Organizing Mixture Networks

Generative Motion Stylization of Cross-structure Characters within Canonical Motion Space

Generative Human Motion Stylization in Latent Space