Abstract:We present an end-to-end method for capturing the dynamics of 3D human characters and translating them for synthesizing new, visually-realistic motion sequences. Conventional methods employ sophisticated, but generic, control approaches for driving the joints of articulated characters, paying little attention to the distinct dynamics of human joint movements. In contrast, our approach attempts to synthesize human-like joint movements by exploiting a biologically-plausible, compact network of spiking neurons that drive joint control in primates and rodents. We adapt the controller architecture by introducing learnable components and propose an evolutionary algorithm for training the spiking neural network architectures and capturing diverse joint dynamics. Our method requires only a few samples for capturing the dynamic properties of a joint's motion and exploits the biologically-inspired, trained controller for its reconstruction. More importantly, it can transfer the captured dynamics to new visually-plausible motion sequences. To enable user-dependent tailoring of the resulting motion sequences, we develop an interactive framework that allows for editing and real-time visualization of the controlled 3D character. We also demonstrate the applicability of our method to real human motion capture data by learning the hand joint dynamics from a gesture dataset and using our framework to reconstruct the gestures with our 3D animated character. The compact architecture of our joint controller emerging from its biologically-realistic design, and the inherent capacity of our evolutionary learning algorithm for parallelization, suggest that our approach could provide an efficient and scalable alternative for synthesizing 3D character animations with diverse and visually-realistic motion dynamics.

Gated Neural Network Framework for Interactive Character Control

Neural Motion Graph.

Novel neural controller for robot manipulator

Optimization algorithm of an artificial neural network-based controller and simulation method for animated virtual idol characters

Deep Neuro-Evolution: Evolving Neural Network for Character Locomotion Controller.

A Reactive and Protective Character Motion Generation Algorithm.

Interactive Animation of Virtual Characters: Application to Virtual Kung-Fu Fighting

Interactive Character Control with Auto-Regressive Motion Diffusion Models

An Interactive Framework for Visually Realistic 3D Motion Synthesis using Evolutionarily-trained Spiking Neural Networks

Real Time Falling Animation with Active and Protective Responses

Optimizing Neural Network As Locomotion Controller With Motion Data

Taming Diffusion Probabilistic Models for Character Control

Dynamic Inference on Graphs using Structured Transition Models

A Method for Character Animation Generation and Control

Multi-Abstractive Neural Controller: An Efficient Hierarchical Control Architecture for Interactive Driving

A Real-Time Control Model for Intelligent Virtual Human

Guided Learning of Control Graphs for Physics-Based Characters

A scalable approach to control diverse behaviors for physically simulated characters

Controllable automatic generation of non‐player characters in 3D anime style

Improved Adaptive Neural Network Control for Humanoid Robot Hand in Workspace