Abstract:We propose a new model-based algorithm solving the inverse rig problem in facial animation retargeting, exhibiting higher accuracy of the fit and sparser, more interpretable weight vector compared to SOTA. The proposed method targets a specific subdomain of human face animation - highly-realistic blendshape models used in the production of movies and video games. In this paper, we formulate an optimization problem that takes into account all the requirements of targeted models. Our objective goes beyond a linear blendshape model and employs the quadratic corrective terms necessary for correctly fitting fine details of the mesh. We show that the solution to the proposed problem yields highly accurate mesh reconstruction even when general-purpose solvers, like SQP, are used. The results obtained using SQP are highly accurate in the mesh space but do not exhibit favorable qualities in terms of weight sparsity and smoothness, and for this reason, we further propose a novel algorithm relying on a MM technique. The algorithm is specifically suited for solving the proposed objective, yielding a high-accuracy mesh fit while respecting the constraints and producing a sparse and smooth set of weights easy to manipulate and interpret by artists. Our algorithm is benchmarked with SOTA approaches, and shows an overall superiority of the results, yielding a smooth animation reconstruction with a relative improvement up to 45 percent in root mean squared mesh error while keeping the cardinality comparable with benchmark methods. This paper gives a comprehensive set of evaluation metrics that cover different aspects of the solution, including mesh accuracy, sparsity of the weights, and smoothness of the animation curves, as well as the appearance of the produced animation, which human experts evaluated.

Accurate and Interpretable Solution of the Inverse Rig for Realistic Blendshape Models with Quadratic Corrective Terms

Refined Inverse Rigging: A Balanced Approach to High-fidelity Blendshape Animation

EasySkinning: Target-oriented Skinning by Mesh Contraction and Curve Editing

A Joint Physical Constraints Avoidance Method for Inverse Kinematics Problem of Redundant Humanoid Manipulator

Animating Scanned Human Models.

High-Quality Mesh Blendshape Generation from Face Videos via Neural Inverse Rendering

Accurate face rig approximation with deep differential subspace reconstruction

A Motion Retargeting Algorithm Based on Model Simplification

A Real-Time Inverse Kinematics Algorithm for Human Motion Modeling

Skinned Motion Retargeting with Preservation of Body Part Relationships

Skinned Motion Retargeting with Dense Geometric Interaction Perception

Facial animation by optimized blendshapes from motion capture data

RAS: A Data‐Driven Rigidity‐Aware Skinning Model For 3D Facial Animation

A Method of Facial Animation Retargeting Based on Motion Capture

REACTO: Reconstructing Articulated Objects from a Single Video

Compressed Skinning for Facial Blendshapes

A Modular Neural Motion Retargeting System Decoupling Skeleton and Shape Perception

Inverse rendering of faces with a 3D morphable model

A Robust and Accurate Approach for Reconstruction of Patient-Specific 3D Bone Models from Sparse Point Sets.

Robust Biharmonic Skinning Using Geometric Fields

Skinned Motion Retargeting with Residual Perception of Motion Semantics & Geometry