Abstract:Comprehending different categories of facial expressions plays a great role in the design of computational model analyzing human perceived and affective state. Authoritative studies have revealed that facial expressions in human daily life are in multiple or co-occurring mental states. However, due to the lack of valid datasets, most previous studies are still restricted to basic emotions with single label. In this paper, we present a novel multi-label facial expression database, RAF-ML, along with a new deep learning algorithm, to address this problem. Specifically, a crowdsourcing annotation of 1.2 million labels from 315 participants was implemented to identify the multi-label expressions collected from social network, then EM algorithm was designed to filter out unreliable labels. For all we know, RAF-ML is the first database in the wild that provides with crowdsourced cognition for multi-label expressions. Focusing on the ambiguity and continuity of blended expressions, we propose a new deep manifold learning network, called Deep Bi-Manifold CNN, to learn the discriminative feature for multi-label expressions by jointly preserving the local affinity of deep features and the manifold structures of emotion labels. Furthermore, a deep domain adaption method is leveraged to extend the deep manifold features learned from RAF-ML to other expression databases under various imaging conditions and cultures. Extensive experiments on the RAF-ML and other diverse databases (JAFFE, CK+$$+$$, SFEW and MMI) show that the deep manifold feature is not only superior in multi-label expression recognition in the wild, but also captures the elemental and generic components that are effective for a wide range of expression recognition tasks.

Recognizing Compound Emotional Expression in Real-World Using Metric Learning Method.

Real-World Facial Expression Recognition Using Metric Learning Method.

Combining 2D Gabor and Local Binary Pattern for Facial Expression Recognition Using Extreme Learning Machine

Efficient Facial Expression Recognition with Representation Reinforcement Network and Transfer Self-Training for Human–Machine Interaction

Real-Time facial expression recognition system based on HMM and feature point localization

Blended Emotion in-the-Wild: Multi-label Facial Expression Recognition Using Crowdsourced Annotations and Deep Locality Feature Learning

DeepEmo: Real-world Facial Expression Analysis Via Deep Learning

Expression Analysis Based on Face Regions in Real-world Conditions.

Expression Analysis Based on Face Regions in Read-world Conditions

Semantic-Rich Facial Emotional Expression Recognition

Compound facial expressions recognition approach using DCGAN and CNN

Reliable Crowdsourcing and Deep Locality-Preserving Learning for Unconstrained Facial Expression Recognition.

An Effective and Efficient Real-Time Facial Expression Recognition Method in the Wild

Emotion Analysis Based on Facial Expression Recognition in Smart Learning Environment

Compound Facial Expression Recognition with Multi-Domain Fusion Expression Based on Adversarial Learning.

Facial Emotion Recognition Based on Brain and Machine Collaborative Intelligence

Face Emotion Recognization Using Dataset Augmentation Based on Neural Network

Emotion Distribution Recognition from Facial Expressions

Reliable Crowdsourcing and Deep Locality-Preserving Learning for Expression Recognition in the Wild

Compound Expression Recognition In-the-wild with AU-assisted Meta Multi-task Learning