Abstract:Facial occlusion is a challenging research topic in facial expression recognition (FER). This has resulted in the need to develop some interesting facial representations and occlusion detection methods in order to extend the FER to uncontrolled environments. It should be noted that most of the previous work focuses on these two issues separately, and on static images. We are thus motivated to propose a complete system consisting of facial representations, occlusion detection, and multiple feature fusion in video sequences. For achieving a robust facial representation, we propose an approach deriving six feature vectors from eyes, nose and mouth components to form a facial representation. These features with temporal cues are generated by the dynamic texture and structural shape feature descriptors. On the other hand, occlusion detection is still mainly realized by the traditional classifiers or model comparison. Recently, sparse representation has been proposed as an efficient method against occlusion, while it is correlated with facial identity in FER, unless using an appropriate facial representation. Thus, we present an evaluation demonstrating that the proposed facial representation is independent of facial identity. Inspired by Mercier et al. (2007), we then exploit the use of the sparse representation and residual statistics to occlusion detection of the image sequences. As concatenating six feature vectors into one causes the curse of dimensionality, we propose multiple feature fusion consisting of fusion module and weight learning. Experimental results on the Extended Cohn-Kanade database and its simulated database demonstrate that our framework outperforms the state-of-the-art methods for FER in normal videos, and especially, in partial occlusion videos.

Robust facial expression recognition under occlusion condition

Robust Facial Expression Recognition Based on Rpca and Adaboost

Robust Face Recognition by Fusion Local Singular Value Feature and Deformable Model

Towards Mask-robust Face Recognition.

Research of Robust Facial Expression Recognition under Facial Occlusion Condition

Local Spatial Continuity Steered Sparse Representation for Occluded Face Recognition

Overcoming Occlusion for Robust Facial Expression Recognition Using Adaptive Dual-Attention Net

An Occluded Facial Expression Recognition Method Based on Sparse Representation

Accurate and robust facial expressions recognition by fusing multiple sparse representation based classifiers.

Robust Modular Linear Regression Based Classification for Face Recognition with Occlusion

A Deep Structure for Facial Expression Recognition under Partial Occlusion

Occlusion-Robust FAU Recognition by Mining Latent Space of Masked Autoencoders

Robust Face Recognition by Constrained Part-based Alignment.

Robust Deep Auto-Encoder For Occluded Face Recognition

Facial Expression Recognition under Partial Occlusion Based on Fusion of Global and Local Features

Occluded Facial Expression Recognition Enhanced Through Privileged Information.

Extracting sparse error of robust PCA for face recognition in the presence of varying illumination and occlusion

Facial Expression Recognition under Partial Occlusion Based on Gabor Filter and Gray-Level Cooccurrence Matrix

Robust PCA for Face Recognition with Occlusion Using Symmetry Information.

Towards a dynamic expression recognition system under facial occlusion

Robust automatic facial expression detection method based on sparse representation plus LBP map