Abstract:Video-based facial expression recognition has received substantial attention over the past decade, while early expression detection (EED) is still a relatively new and challenging problem. The goal of EED is to identify an expression as quickly as possible after the expression starts and before it ends. This timely ability has many potential applications, ranging from human-computer interaction to security. The max-margin early event detector (MMED) is a well-known ranking model for early event detection. It can achieve competitive EED performance but suffers from several critical limitations: 1) MMED lacks flexibility in extracting useful information for segment comparison, which leads to poor performance in exploring the ranking relation between segment pairs; 2) the training process is slow due to the large number of constraints, and the memory requirement is also usually hard to satisfy; and 3) MMED is linear in nature, and hence may not be appropriate for data in a nonlinear feature space. To overcome these limitations, we propose an online multi-instance learning (MIL) framework for EED. In particular, the MIL technique is first introduced to generalize MMED, resulting in the proposed MIL-based EED (MIED), which is more general and flexible than MMED, since various instance construction and combination strategies can be adopted. To accelerate the training process, we reformulate MIED in the online setting and develop online multi-instance learning framework for EED (OMIED). To further exploit the nonlinear structure of the data distribution, we incorporate the kernel methods in OMIED, which results in the proposed online kernel multi-instance learning for early expression detection. Experiments on two popular and one challenging video-based expression data sets demonstrate both the efficiency and effectiveness of the proposed methods.

Expression Recognition Using 2D Embedded HMM.

Dynamic hand gesture recognition using hidden Markov models

Early Facial Expression Recognition Using Hidden Markov Models

Combining 2D Gabor and Local Binary Pattern for Facial Expression Recognition Using Extreme Learning Machine

Real-Time facial expression recognition system based on HMM and feature point localization

Expression Recognition Based on EPCA and SVM

Non-frontal view facial expression recognition based on ergodic hidden Markov model supervectors

Efficient Facial Expression Recognition with Representation Reinforcement Network and Transfer Self-Training for Human–Machine Interaction

Automatic facial expression recognition on a single 3D face by exploring shape deformation.

The Human Facial Combined Expression Recognition System

Expression recognition from video using a coupled hidden Markov model

Expression Recognition Based on Human Emotion-Attention Circuit

Early Expression Detection Via Online Multi-Instance Learning with Nonlinear Extension

Research on Dynamic Facial Expressions Recognition

Emotion Recognition via Environmental Context and Human Body

A Navel Facial Expression Recognition Method Based On Extreme Learning Machine

A Facial Expression Emotion Recognition Based Human-robot Interaction System

Research on the embedded system of facial expression recognition based on HMM

Subtle expression recognition-a comprehensive approach

Multi-Instance Hidden Markov Model For Facial Expression Recognition

Expression Recognition Based on Genetic Algorithm and SVM.