Abstract:Facial expression recognition (FER) plays an important role in cognitive psychology research. In FER studies, deep convolutional neural networks (CNNs) and attention mechanisms have shown great advantages over traditional techniques. Although many attention-based CNN models have been developed, most of them are proposed to deal with grayscale images. For color images, these network models usually take their grayscale versions as the input or convert them to grayscale data by the first hidden layer. Furthermore, their attention modules focus on only grayscale information. This mechanism does not fully consider the correlation between color channels, which will degrade the performance of color FER. To alleviate these shortcomings, we introduce quaternion techniques to CNNs and propose a quaternion CNN integrated with an attention mechanism (QA-CNN) for color FER. The proposed QA-CNN takes into account not only the correlation between color channels but also expressional attention. The new attention mechanism is based on a multidirectional quaternion Gabor filter. Besides, the proposed QA-CNN reduces the number of network parameters by 75% compared with the real-valued CNN with the same structure. Experimental results on three widely used data sets demonstrate the effectiveness of the proposed QA-CNN by showing clear performance improvements over other state-of-the-art FER methods.

Quaternion Based Information Parallel Fusion and Its Application in Color Face Detection

A new facial feature based on the fusion of texture and shape characteristics

Iris-Face Fusion and Security Analysis Based on Fisher Discriminant

Face Detection Based on Vector Quantization in Color Images

Quasi Non-Negative Quaternion Matrix Factorization with Application to Color Face Recognition

Facial Target Detection and Recognition Algorithm Based on Information Fusion

Color Facial Expression Recognition by Quaternion Convolutional Neural Network With Gabor Attention

Color Face Recognition using High-Dimension Quaternion-based Adaptive Representation

An Approach of Face Detection Based on Multicomponent Information Fusion

Quaternion Discrete Cosine Transform and its Application in Color Template Matching

Quaternion Scalar and Vector Norm Decomposition: Quaternion PCA for Color Face Recognition

Joint diagonalization for a pair of Hermitian quaternion matrices and applications to color face recognition

Generalized Two-Dimensional Quaternion Principal Component Analysis with Weighting for Color Image Recognition

Quaternion Convolutional Neural Networks

The Realistic Fusion of Multi-spectral Images

Robust face recognition using quaternion interval type II fuzzy logic–based feature extraction on colour images

Kernel Quaternion Principal Component Analysis and Its Application in RGB-D Object Recognition.

Information Fusion in Face Identification.

Content-based face image retrieval using quaternion based local diagonal extreme value pattern

A Quaternion Phase-Only Correlation Algorithm for Color Images.

Quaternion Convolutional Neural Networks for Heterogeneous Image Processing