Abstract:In the construction of smart cities, facial expression analysis plays a crucial role. It can be used in traffic monitoring systems to alleviate traffic pressure by analyzing the emotional states of drivers and passengers. In the field of smart healthcare, it can provide more precise treatment and services to patients. In the realm of social entertainment, it can offer more intelligent and personalized interactions. In summary, the application of emotion computing technology will play an increasingly significant role in the development of smart cities in the future. In the task of dynamic facial expression recognition (DFER), analyzing the spatial–temporal features of video sequences has become a common research approach. However, facial expression sequences often contain a significant number of neutral frames and noisy frames, potentially increasing computational costs and reducing performance. Effectively extracting key frames for spatial–temporal feature analysis is a critical aspect of dynamic facial expression recognition. To address this issue, we proposed a sampling-wise dynamic facial expression recognition via frame-Sequence contrastive learning method, called SW-FSCL. The SW-FSCL method aims to improve the performance of DFER by using intelligent dual-stream sampling strategies and frame-sequence contrastive learning, extract key frame and reduce the impact of neutral frames and noisy frames. We proposed a key frame proposal (KFP) block to analyze the spatial–temporal features of sequences, calculating weight ratios for key frame extraction. Due to potential information loss in long sequences, we introduce a temporal aggregation (TA) block to prevent data loss and ensure the integrity of temporal information. The experimental results provide compelling evidence that the proposed approach not only outperforms all current state-of-the-art algorithms on two widely-used benchmark datasets (DFEW, FERV39k), but also visualization results produces insights into the interpretability of the SW-FSCL method.

Facial Expression Recognition Based On Group Domain Random Frame Extraction

A New Deep Learning Method for Multi-label Facial Expression Recognition Based on Local Constraint Features

A discriminative multiscale feature extraction network for facial expression recognition in the wild

Facial Expression Recognition With Fusion Features Extracted From Salient Facial Areas

Empower Smart Cities with Sampling-Wise Dynamic Facial Expression Recognition Via Frame-Sequence Contrastive Learning

A Joint Local Spatial and Global Temporal CNN-Transformer for Dynamic Facial Expression Recognition

Facial Expression Recognition Via Region-Based Convolutional Fusion Network.

Observe Finer to Select Better: Learning Key Frame Extraction Via Semantic Coherence for Dynamic Facial Expression Recognition in the Wild

Spatio-Temporal Facial Expression Recognition Using Convolutional Neural Networks and Conditional Random Fields

Facial Expression Recognition in the Wild Based on Multimodal Texture Features

A Cascade Attention Based Facial Expression Recognition Network by Fusing Multi-Scale Spatio-Temporal Features

Facial Expression Recognition Based on Deep Evolutional Spatial-Temporal Networks

Feature Decomposition and Reconstruction Learning for Effective Facial Expression Recognition

Disentangled-region Non-Local Neural Network for Facial Expression Recognition

A New Framework Combining Local-Region Division and Feature Selection for Micro-Expressions Recognition.

Facial Micro-expression Recognition Using Enhanced Temporal Feature-Wise Model

Facial expression recognition with convolutional neural networks via a new face cropping and rotation strategy

Geometric-Convolutional Feature Fusion Based On Learning Propagation For Facial Expression Recognition

Enhanced Spatial-Temporal Learning Network for Dynamic Facial Expression Recognition