Abstract:Previous research on Facial Expression Recognition (FER) assisted by facial landmarks mainly focused on single-task learning or hard-parameter sharing based multi-task learning. However, soft-parameter sharing based methods have not been explored in this area. Therefore, this paper adopts Facial Landmark Detection (FLD) as the auxiliary task and explores new multi-task learning strategies for FER. First, three classical multi-task structures, including Hard-Parameter Sharing (HPS), Cross-Stitch Network (CSN), and Partially Shared Multi-task Convolutional Neural Network (PS-MCNN), are used to verify the advantages of multi-task learning for FER. Then, we propose a new end-to-end Co-attentive Multi-task Convolutional Neural Network (CMCNN), which is composed of the Channel Co-Attention Module (CCAM) and the Spatial Co-Attention Module (SCAM). Functionally, the CCAM generates the channel co-attention scores by capturing the inter-dependencies of different channels between FER and FLD tasks. The SCAM combines the max- and average-pooling operations to formulate the spatial co-attention scores. Finally, we conduct extensive experiments on four widely used benchmark facial expression databases, including RAF, SFEW2, CK+, and Oulu-CASIA. Extensive experimental results show that our approach achieves better performance than single-task and multi-task baselines, fully validating multi-task learning’s effectiveness and generalizability 1 1 Codes and detailed instructions can be available at https://github.com/thuiar/cmcnn. .

Learning a Multi-Center Convolutional Network for Unconstrained Face Alignment.

Deep Multi-Center Learning for Face Alignment

Multi-task Convolution Network for Face Alignment

End-to-End Spatial Transform Face Detection and Recognition

SADRNet: Self-Aligned Dual Face Regression Networks for Robust 3D Dense Face Alignment and Reconstruction

Real-Time Facial Landmark Detection by Attention-driven Lightweight Network

Facial Landmark Detection Via Cascade Multi-Channel Convolutional Neural Network

Face Alignment by Deep Convolutional Network with Adaptive Learning Rate.

Cascaded Deep Convolutional Neural Network for Robust Face Alignment.

Multistage Model for Robust Face Alignment Using Deep Neural Networks

Joint Multi-View Face Alignment in the Wild

Multi-face detection and alignment using multiple kernels

Region-Based Face Alignment With Convolution Neural Network Cascade

Deep Shape Constrained Network for Robust Face Alignment.

Layer-wise Supervised Neural Network for Face Alignment with Multi-Task Regularization

Effective face landmark localization via single deep network

Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks

Co-attentive multi-task convolutional neural network for facial expression recognition

Multi-scale Attention Guided Network for End-to-end Face Alignment and Recognition

Joint Face Detection and Alignment Using Focal Loss-Based Multi-task Convolutional Neural Networks

Unconstrained Face Alignment Without Face Detection.