Abstract:We present a new system that applies an example-based learning method to learn facial motion patterns from a video sequence of individual facial behavior such as lip motion and facial expressions, and using that to create vivid three-dimensional (3-D) face animation according to the definition of MPEG-4 face animation parameters. The system consists of three key modules, face tracking, pattern learning, and face animation. In face tracking, to reduce the complexity of the tracking process, a novel coarse-to-fine strategy combined with a Kalman filter is proposed for localizing key facial landmarks in each image of the video. The landmarks' sequence is normalized into a visual feature matrix and then fed to the next step of process. In pattern learning, in the pretraining stage, the parameters of the camera that took the video are requested with the training video data so the system can estimate the basic mapping from a normalized two-dimensional (2-D) visual feature matrix to the representation in 3-D MPEG-4 face animation parameter space, in assistance with the computer vision method. In the practice stage, considering that in most cases camera parameters are not provided with video data, the system uses machine learning technology to complement the incomplete 3-D information for the mapping that information is needed in face orientation presentation. The example-based learning in this system integrates several methods including clustering, HMM, and ANN to make a better conversion from a 2-D to 3-D model and better estimation of incomplete 3-D information for good mapping; this will be used to drive face animation thereafter. In face animation, the system can synthesize face animation following any type of face motion in video. Experiments show that our system produces more vivid face motion animation, compared to other early systems.

ASAP: animation system for agent-based presentations

AesopAgent: Agent-driven Evolutionary System on Story-to-Video Production

A novel animation authoring framework for the virtual teacher performing experiment in mixed reality

Interactive Animation of Virtual Characters: Application to Virtual Kung-Fu Fighting

Animated Pedagogical Agents Enhance Learning Outcomes and Brain Activity During Learning.

Automatic Generation of Human Animation Based on Motion Programming: Virtual Humans and Social Agents

Text-to-movie authoring of anatomy lessons

Anim-Director: A Large Multimodal Model Powered Agent for Controllable Animation Video Generation

An embodiment effect in computer-based learning with animated pedagogical agents.

Automatic Generation of Human Animation Based on Motion Programming.

Learning and Synthesizing Mpeg-4 Compatible 3-D Face Animation from Video Sequence

Interactive Facial-Geometric-Feature Animation For Generating Expressions Of Novel Faces

Real-time Expressive Avatar Animation Generation Based on Monocular Videos.

AnimateAnything: Consistent and Controllable Animation for Video Generation

2D animation comic character action generation technology based on biomechanics simulation and artificial intelligence

Write‐An‐Animation: High‐level Text‐based Animation Editing with Character‐Scene Interaction

Research on 3D animation character design based on multimedia interaction

Improving Visual Awareness by Real-Time 2D Facial Animation for Ubiquitous Collaboration

From Data to Story: Towards Automatic Animated Data Video Creation with LLM-based Multi-Agent Systems

Towards smarter animated embodied agents

Lecture Video Enhancement and Editing by Integrating Posture, Gesture, and Text