Abstract:Complex emotion recognition is a cognitive task that has so far eluded the same excellent performance of other tasks that are at or above the level of human cognition. Emotion recognition through facial expressions is particularly difficult due to the complexity of emotions expressed by the human face. For a machine to approach the same level of performance in this domain as a human, it may need to synthesise knowledge and understand new concepts in real-time as humans do. Humans are able to learn new concepts using only few examples, by distilling the important information from memories and discarding the rest. Similarly, continual learning methods learn new classes whilst retaining the knowledge of known classes, whilst few-shot learning methods are able to learn new classes using very few training examples. We propose a novel continual learning method inspired by human cognition and learning that can accurately recognise new compound expression classes using few training samples, by building on and retaining its knowledge of basic expression classes. Using GradCAM visualisations, we demonstrate the relationship between basic and compound facial expressions, which our method leverages through knowledge distillation and a novel Predictive Sorting Memory Replay. Our method achieves the current state-of-the-art in continual learning for complex facial expression recognition with 74.28% Overall Accuracy on new classes. We also demonstrate that using continual learning for complex facial expression recognition achieves far better performance than non-continual learning methods, improving on state-of-the-art non-continual learning methods by 13.95%. To the best of our knowledge, our work is also the first to apply few-shot learning to complex facial expression recognition, achieving the state-of-the-art with 100% accuracy using a single training sample for each expression class.

Learning of Complicate Facial Expression Categories

A Transfer Learning Approach to Compound Facial Expression Recognition

Antigen presentation in brain: brain endothelial cells are poor stimulators of T-cell proliferation.

Compound facial expressions recognition approach using DCGAN and CNN

Facial expression recognition with convolutional neural networks via a new face cropping and rotation strategy

Two-pathway attention network for real-time facial expression recognition

Cross-Layer Contrastive Learning of Latent Semantics for Facial Expression Recognition

Automatic Facial Expression Recognition Based on a Deep Convolutional-Neural-network Structure

Learning Associative Representation for Facial Expression Recognition.

Adaptively Learning Facial Expression Representation via C-F Labels and Distillation

A discriminative multiscale feature extraction network for facial expression recognition in the wild

Attention mechanism-based CNN for facial expression recognition

Facial Expression Recognition Method Based on Improved VGG Convolutional Neural Network

An Efficient Channel Attention CNN for Facial Expression Recognition

Hybrid Attention-Aware Learning Network for Facial Expression Recognition in the Wild

Facial expression recognition through multi-level features extraction and fusion

Learning Informative and Discriminative Features for Facial Expression Recognition in the Wild

Relation-aware Network for Facial Expression Recognition

Complex Facial Expression Recognition Using Deep Knowledge Distillation of Basic Features

Facial Expression Recognition Methods in the Wild Based on Fusion Feature of Attention Mechanism and LBP

Facial Expression Recognition System Based On Deep Residual Fusion Neural Network