Abstract:Affect recognition based on subjects' facial expressions has been a topic of major research in the attempt to generate machines that can understand the way subjects feel, act and react. In the past, due to the unavailability of large amounts of data captured in real-life situations, research has mainly focused on controlled environments. However, recently, social media and platforms have been widely used. Moreover, deep learning has emerged as a means to solve visual analysis and recognition problems. This paper exploits these advances and presents significant contributions for affect analysis and recognition in-the-wild. Affect analysis and recognition can be seen as a dual knowledge generation problem, involving: i) creation of new, large and rich in-the-wild databases and ii) design and training of novel deep neural architectures that are able to analyse affect over these databases and to successfully generalise their performance on other datasets. The paper focuses on large in-the-wild databases, i.e., Aff-Wild and Aff-Wild2 and presents the design of two classes of deep neural networks trained with these databases. The first class refers to uni-task affect recognition, focusing on prediction of the valence and arousal dimensional variables. The second class refers to estimation of all main behavior tasks, i.e. valence-arousal prediction; categorical emotion classification in seven basic facial expressions; facial Action Unit detection. A novel multi-task and holistic framework is presented which is able to jointly learn and effectively generalize and perform affect recognition over all existing in-the-wild databases. Large experimental studies illustrate the achieved performance improvement over the existing state-of-the-art in affect recognition.

ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Multi-Task Learning Challenges

ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Emotional Reaction Intensity Estimation Challenges

7th ABAW Competition: Multi-Task Learning and Compound Expression Recognition

Analysing Affective Behavior in the First ABAW 2020 Competition

The 6th Affective Behavior Analysis in-the-wild (ABAW) Competition

Multi-Task Learning for Emotion Descriptors Estimation at the fourth ABAW Challenge

ABAW : Facial Expression Recognition in the wild

Affective Behaviour Analysis via Integrating Multi-Modal Knowledge

Facial Affective Behavior Analysis Method for 5th ABAW Competition

Affective Behaviour Analysis via Progressive Learning

Affect Analysis in-the-wild: Valence-Arousal, Expressions, Action Units and a Unified Framework

An Effective Ensemble Learning Framework for Affective Behaviour Analysis

Affective Expression Analysis in-the-wild using Multi-Task Temporal Statistical Deep Learning Model

Prior Aided Streaming Network for Multi-task Affective Recognitionat the 2nd ABAW2 Competition

Technical Report for Valence-Arousal Estimation in ABAW2 Challenge

A Multi-term and Multi-task Analyzing Framework for Affective Analysis in-the-wild

HSEmotion Team at the 6th ABAW Competition: Facial Expressions, Valence-Arousal and Emotion Intensity Prediction

SUN Team's Contribution to ABAW 2024 Competition: Audio-visual Valence-Arousal Estimation and Expression Recognition

A Unified Approach to Facial Affect Analysis: the MAE-Face Visual Representation.

Facial Affect Recognition based on Transformer Encoder and Audiovisual Fusion for the ABAW5 Challenge

Facial Affect Analysis: Learning from Synthetic Data & Multi-Task Learning Challenges