Abstract:Characterizing subtle facial movements from videos is one of the most intensive topics in computer vision research. It is, however, challenging, since (1) the intensity of subtle facial muscle movement is usually low, (2) the duration may be transient, and (3) datasets containing spontaneous subtle movements with reliable annotations are painful to obtain and often of small sizes. This article is targeted at addressing these problems for characterizing subtle facial movements from both the aspects of motion elucidation and description. First, we propose an efficient method for elucidating hidden and repressed movements to make them easier to get noticed. We explore the feasibility of linearizing motion magnification and temporal interpolation, which is obscured by the architecture of existing methods. On this basis, we propose a consolidated framework, termed MOTEL, to expand temporal duration and amplify subtle facial movements simultaneously. Second, we make our contribution to dynamic description. One major challenge is to capture the intrinsic temporal variations caused by movements and omit extrinsic ones caused by different individuals and various environments. To diminish the influences of such extrinsic diversity, we propose the tangent delta descriptor to characterize the dynamics of short-term movements using the differences between points on the tangent spaces to the manifolds, rather than the points themselves. We then relax the trajectory-smooth assumption of the conventional manifold-based trajectory modeling methods and incorporate the tangent delta descriptor with the sequential inference approaches to cover the period of facial movements. The proposed motion modeling approach is validated by a series of experiments on publicly available datasets in the tasks of micro-expression recognition and visual speech recognition.

Subtle Facial Expression Synthesis Using Motion Manifold Embedding and Nonlinear Decomposable Generative Models

Video Tracked Facial Expression Animation

Characterizing Subtle Facial Movements Via Riemannian Manifold

Video-driven state-aware facial animation

Dynamic Facial Expression Generation on Hilbert Hypersphere With Conditional Wasserstein Generative Adversarial Nets

High Resolution Acquisition, Learning And Transfer Of Dynamic 3-D Facial Expressions

Learning Expressionlets Via Universal Manifold Model for Dynamic Facial Expression Recognition

Learning Expressionlets on Spatio-temporal Manifold for Dynamic Facial Expression Recognition

Subtle Facial Expression Modeling With Vector Field Decomposition

Facial Expression Synthesis Using Manifold Learning and Belief Propagation

Expressive facial animation synthesis by learning speech coarticulation and expression spaces

Facial Expression Animation by Landmark Guided Residual Module

Manifold-enhanced CycleGAN for Facial Expression Synthesis

Hierarchical facial expression animation by motion capture data

Dynamic Facial Expression Synthesis Driven by Deformable Semantic Parts.

Facial Expression Synthesis Based on Facial Component Model.

Data-driven facial expression synthesis via Laplacian deformation

Facial Expression Recognition on Multiple Manifolds

3D Facial Expression Synthesis Based on Nonlinear Co-learning

Extended facial expression synthesis using statistical appearance model

Dynamic Facial Expression Recognition Using A Bayesian Temporal Manifold Model