Abstract:Person-dependent facial expression recognition has received considerable research attention in recent years. Unfortunately, different identities can adversely influence recognition accuracy, and the recognition task becomes challenging. Other adverse factors, including limited training data and improper measures of facial expressions, can further contribute to the above dilemma. To solve these problems, a novel identity-aware method is proposed in this study. Furthermore, this study also represents the first attempt to fulfill the challenging person-dependent facial expression recognition task based on deep metric learning and facial image synthesis techniques. Technically, a StarGAN is incorporated to synthesize facial images depicting different but complete basic emotions for each identity to augment the training data. Then, a deep-convolutional-neural-network-based network is employed to automatically extract latent features from both real facial images and all synthesized facial images. Next, a Mahalanobis metric network trained based on extracted latent features outputs a learned metric that measures facial expression differences between images, and the recognition task can thus be realized. Extensive experiments based on several well-known publicly available datasets are carried out in this study for performance evaluations. Person-dependent datasets, including CK+, Oulu (all 6 subdatasets), MMI, ISAFE, ISED, etc., are all incorporated. After comparing the new method with several popular or state-of-the-art facial expression recognition methods, its superiority in person-dependent facial expression recognition can be proposed from a statistical point of view.

Facial Expression Synthesis and Recognition with Pre-Trained StyleGAN

Expression Conditional Gan for Facial Expression-to-Expression Translation.

Manifold-enhanced CycleGAN for Facial Expression Synthesis

How to Boost Face Recognition with StyleGAN?

Unconstrained Facial Expression Transfer using Style-based Generator

A Facial Expression Synthesis Method Based on Generative Adversarial Network.

Specialized discriminators for style consistency in facial expression synthesis

Joint Expression Synthesis and Representation Learning for Facial Expression Recognition

Joint Deep Learning of Facial Expression Synthesis and Recognition

Identity-aware Facial Expression Recognition via Deep Metric Learning based on Synthesized Images

Two Birds with One Stone: Iteratively Learn Facial Attributes with GANs.

ApprGAN: Appearance-Based Generative Adversarial Network for Facial Expression Synthesis

Comp-GAN

ApprGAN: appearance‐based GAN for facial expression synthesis

Facial Expression Synthesis by U-Net Conditional Generative Adversarial Networks

Two Birds with One Stone: Transforming and Generating Facial Images with Iterative GAN

Facial Landmarks and Expression Label Guided Photorealistic Facial Expression Synthesis

Multi-expression Generative Adversarial Networks for Facial Expression Synthesis

Cross-generating GAN for Facial Identity Preserving

ExprGAN: Facial Expression Editing With Controllable Expression Intensity