Abstract:Facial expression is highly correlated with the facial motion. According to whether the temporal information of facial motion is used or not, the facial expression features can be classified as static and dynamic features. The former, which mainly includes the geometric features and appearance features, can be extracted by convolution or other learning filters; the latter, which are aimed to model the dynamic properties of facial motion, can be calculated through optical flow or other methods, respectively. When 3D convolutional neural networks (CNNs) are introduced, the extraction of two different types of features mentioned above becomes easy. In this paper, one 3D CNN architecture is presented to learn the static and dynamic features from facial image sequences and extract high-level dynamic features from optical flow sequences. Two types of dense optical flow, which contain the tracking information of facial muscle movement, are calculated according to different image pair construction methods. One is the common optical flow, and the other is an enhanced optical flow which is called accumulative optical flow. Four components of each type of optical flow are used in experiments. Three databases, two acted databases and one nearly realistic database, are selected to conduct the experiments. The experiments on the two acted databases achieve state-of-the-art accuracy, and indicate that the vertical component of optical flow has an advantage over other components in recognizing facial expression. The experimental results on the three selected databases show that more discriminative features can be learned from image sequences than from optical flow or accumulative optical flow sequences, and the accumulative optical flow contains more motion information than optical flow if the frame distance of the image pairs used to calculate them is not too large.

Facial Expression Recognition Based on Optical Flow Techniques for Non-rigid Motion Analysis

Dynamic Facial Expression Recognition Based on Optical Flow and Geometric Features

Facial Optical Flow Estimation Via Neural Non-Rigid Registration

Analysis and Application of the Facial Expression Motions Based on Eigen-Flow

A method for recognizing facial expression intensity based on facial muscle variations

Real-Time facial expression recognition system based on HMM and feature point localization

Research on Micro-Expression Spotting Method Based on Optical Flow Features

Expression Intensity Recognition Based on Multilayer Hybrid Classifier.

Facial Expression Recognition Based on the Fusion of Spatio-temporal Features in Video Sequences

Facial Expression Recognition Based on Spatio-Temporal Interest Points for Depth Sequences

Video-Based Facial Expression Recognition Using Histogram Sequence of Local Gabor Binary Patterns from Three Orthogonal Planes

Video Tracked Facial Expression Animation

What Happens in Face During a Facial Expression? Using Data Mining Techniques to Analyze Facial Expression Motion Vectors

A Main Directional Mean Optical Flow Feature for Spontaneous Micro-Expression Recognition.

Learning Deep Facial Expression Features from Image and Optical Flow Sequences Using 3D CNN

A Visualization Method of Facial Expression Deformation Based on Pore-Scale Facial Feature Matching

Efficient Micro-Expression Spotting Based on Main Directional Mean Optical Flow Feature

Magnifying Subtle Facial Motions for Effective 4D Expression Recognition

An Efficient Optical Flow Based Motion Detection Method for Non-stationary Scenes

Microexpression Identification and Categorization Using a Facial Dynamics Map

3D facial expression tracking using a monocular RGB camera