Abstract:Creating a large and natural facial expression database is a prerequisite for facial expression analysis and classification. It is, however, not only time consuming but also difficult to capture an adequately large number of spontaneous facial expression images and their meanings because no standard, uniform, and exact measurements are available for database collection and annotation. Thus, comprehensive first-hand data analyses of a spontaneous expression database may provide insight for future research on database construction, expression recognition, and emotion inference. This paper presents our analyses of a multimodal spontaneous facial expression database of natural visible and infrared facial expressions (NVIE). First, the effectiveness of emotion-eliciting videos in the database collection is analyzed with the mean and variance of the subjects' self-reported data. Second, an interrater reliability analysis of raters' subjective evaluations for apex expression images and sequences is conducted using Kappa and Kendall's coefficients. Third, we propose a matching rate matrix to explore the agreements between displayed spontaneous expressions and felt affective states. Lastly, the thermal differences between the posed and spontaneous facial expressions are analyzed using a paired-samples t-test. The results of these analyses demonstrate the effectiveness of our emotion-inducing experimental design, the gender difference in emotional responses, and the coexistence of multiple emotions/expressions. Facial image sequences are more informative than apex images for both expression and emotion recognition. Labeling an expression image or sequence with multiple categories together with their intensities could be a better approach than labeling the expression image or sequence with one dominant category. The results also demonstrate both the importance of facial expressions as a means of communication to convey affective states and the diversity of the displayed ma- ifestations of felt emotions. There are indeed some significant differences between the temperature difference data of most posed and spontaneous facial expressions, many of which are found in the forehead and cheek regions.

Facial Expression Analysis Across Databases

Cgan Based Facial Expression Recognition for Human-Robot Interaction

Cross-Database Facial Expression Recognition Based on Fine-Tuned Deep Convolutional Network

Evaluation of Data Augmentation Techniques for Facial Expression Recognition Systems

A hierarchical algorithm with multi-feature fusion for facial expression recognition

Casme Ii: an Improved Spontaneous Micro-Expression Database and the Baseline Evaluation

Expression Analysis Based on Face Regions in Real-world Conditions.

Combining 2D Gabor and Local Binary Pattern for Facial Expression Recognition Using Extreme Learning Machine

Cross-database non-frontal facial expression recognition based on transductive deep transfer learning

Cross-database facial expression recognition based on hybrid improved unsupervised domain adaptation

Video-Based Facial Expression Recognition Using Histogram Sequence of Local Gabor Binary Patterns from Three Orthogonal Planes

Human and machine validation of 14 databases of dynamic facial expressions

A 4D spontaneous micro‐expression database: Establishment and evaluation

Analyses of a Multimodal Spontaneous Facial Expression Database

Expression Analysis Based on Face Regions in Read-world Conditions

A Natural Visible and Infrared Facial Expression Database for Expression Recognition and Emotion Inference

Multi-Representation Joint Dynamic Domain Adaptation Network for Cross-Database Facial Expression Recognition

Facial Expression Recognition Using Hybrid Features of Pixel and Geometry

Towards East Asian Facial Expression Recognition in the Real World: A New Database and Deep Recognition Baseline

Survey on Facial Expression Recognition: History, Applications, and Challenges

Sequence-based bias analysis of spontaneous facial expression databases