Abstract:Person-dependent facial expression recognition has received considerable research attention in recent years. Unfortunately, different identities can adversely influence recognition accuracy, and the recognition task becomes challenging. Other adverse factors, including limited training data and improper measures of facial expressions, can further contribute to the above dilemma. To solve these problems, a novel identity-aware method is proposed in this study. Furthermore, this study also represents the first attempt to fulfill the challenging person-dependent facial expression recognition task based on deep metric learning and facial image synthesis techniques. Technically, a StarGAN is incorporated to synthesize facial images depicting different but complete basic emotions for each identity to augment the training data. Then, a deep-convolutional-neural-network-based network is employed to automatically extract latent features from both real facial images and all synthesized facial images. Next, a Mahalanobis metric network trained based on extracted latent features outputs a learned metric that measures facial expression differences between images, and the recognition task can thus be realized. Extensive experiments based on several well-known publicly available datasets are carried out in this study for performance evaluations. Person-dependent datasets, including CK+, Oulu (all 6 subdatasets), MMI, ISAFE, ISED, etc., are all incorporated. After comparing the new method with several popular or state-of-the-art facial expression recognition methods, its superiority in person-dependent facial expression recognition can be proposed from a statistical point of view.

Multimodal Face Synthesis From Visual Attributes

Facial Synthesis from Visual Attributes via Sketch using Multi-Scale Generators

Attribute-Guided Sketch Generation

Multimodal-driven Talking Face Generation, Face Swapping, Diffusion Model

Face Synthesis from Visual Attributes via Sketch using Conditional VAEs and GANs

Recognizing Facial Sketches by Generating Photorealistic Faces Guided by Descriptive Attributes

Conditional Face Synthesis for Data Augmentation.

Towards Open-Set Identity Preserving Face Synthesis

Recognizing Minimal Facial Sketch by Generating Photorealistic Faces with the Guidance of Descriptive Attributes

Synthesis of High-Quality Visible Faces from Polarimetric Thermal Faces using Generative Adversarial Networks

From Eyes to Face Synthesis: a New Approach for Human-Centered Smart Surveillance

CMOS-GAN: Semi-Supervised Generative Adversarial Model for Cross-Modality Face Image Synthesis

Biphasic Face Photo-Sketch Synthesis via Semantic-Driven Generative Adversarial Network with Graph Representation Learning

Generative Adversarial Network-based Synthesis of Visible Faces from Polarimetric Thermal Faces

Multi-expression Generative Adversarial Networks for Facial Expression Synthesis

Quality Guided Sketch-to-Photo Image Synthesis

Joint Sketch-Attribute Learning for Fine-Grained Face Synthesis.

Identity-aware Facial Expression Recognition via Deep Metric Learning based on Synthesized Images

Generating Face Images with Attributes for Free.

Face Photo–sketch Synthesis Via Intra-Domain Enhancement

Generating Photographic Faces From the Sketch Guided by Attribute Using GAN.