Abstract:We present an end-to-end method for capturing the dynamics of 3D human characters and translating them for synthesizing new, visually-realistic motion sequences. Conventional methods employ sophisticated, but generic, control approaches for driving the joints of articulated characters, paying little attention to the distinct dynamics of human joint movements. In contrast, our approach attempts to synthesize human-like joint movements by exploiting a biologically-plausible, compact network of spiking neurons that drive joint control in primates and rodents. We adapt the controller architecture by introducing learnable components and propose an evolutionary algorithm for training the spiking neural network architectures and capturing diverse joint dynamics. Our method requires only a few samples for capturing the dynamic properties of a joint's motion and exploits the biologically-inspired, trained controller for its reconstruction. More importantly, it can transfer the captured dynamics to new visually-plausible motion sequences. To enable user-dependent tailoring of the resulting motion sequences, we develop an interactive framework that allows for editing and real-time visualization of the controlled 3D character. We also demonstrate the applicability of our method to real human motion capture data by learning the hand joint dynamics from a gesture dataset and using our framework to reconstruct the gestures with our 3D animated character. The compact architecture of our joint controller emerging from its biologically-realistic design, and the inherent capacity of our evolutionary learning algorithm for parallelization, suggest that our approach could provide an efficient and scalable alternative for synthesizing 3D character animations with diverse and visually-realistic motion dynamics.

Real-Time Synthesis of 3d Animations by Learning Parametric Gaussians Using Self-Organizing Mixture Networks

Key-styling: learning motion style for real-time synthesis of 3D animation

Key-styling: learning motion style for real-time synthesis of 3D animation: Research Articles

Supervised Learning of Motion Style for Real-time Synthesis of 3D Character Animations

Learning Style-Directed Dynamics Of Human Motion For Automatic Motion Synthesis

The SOMN-HMM Model and Its Application to Automatic Synthesis of 3D Character Animations

Animatable 3D Gaussians for High-fidelity Synthesis of Human Motions.

Real-Time 3D Human Cartoon Animation System

3D Human Motion Synthesis Based on Nonlinear Manifold Learning

Editing and Synthesis of 3D Facial Animation by Motion Propagation and Isomap Analysis

Human Gaussian Splatting: Real-time Rendering of Animatable Avatars

Animatable and Relightable Gaussians for High-fidelity Human Avatar Modeling

Real-Time Style Modelling of Human Locomotion via Feature-Wise Transformations and Local Motion Phases

Animatable Gaussians: Learning Pose-dependent Gaussian Maps for High-fidelity Human Avatar Modeling

Motion Synthesis Using Style-Editable Inverse Kinematics

GAST: Sequential Gaussian Avatars with Hierarchical Spatio-temporal Context

An Interactive Framework for Visually Realistic 3D Motion Synthesis using Evolutionarily-trained Spiking Neural Networks

Autoregressive Stylized Motion Synthesis with Generative Flow

Gaussians-to-Life: Text-Driven Animation of 3D Gaussian Splatting Scenes

Animatable 3D Gaussian: Fast and High-Quality Reconstruction of Multiple Human Avatars

Topology-aware Human Avatars with Semantically-guided Gaussian Splatting