Abstract:Creating a large and natural facial expression database is a prerequisite for facial expression analysis and classification. It is, however, not only time consuming but also difficult to capture an adequately large number of spontaneous facial expression images and their meanings because no standard, uniform, and exact measurements are available for database collection and annotation. Thus, comprehensive first-hand data analyses of a spontaneous expression database may provide insight for future research on database construction, expression recognition, and emotion inference. This paper presents our analyses of a multimodal spontaneous facial expression database of natural visible and infrared facial expressions (NVIE). First, the effectiveness of emotion-eliciting videos in the database collection is analyzed with the mean and variance of the subjects' self-reported data. Second, an interrater reliability analysis of raters' subjective evaluations for apex expression images and sequences is conducted using Kappa and Kendall's coefficients. Third, we propose a matching rate matrix to explore the agreements between displayed spontaneous expressions and felt affective states. Lastly, the thermal differences between the posed and spontaneous facial expressions are analyzed using a paired-samples t-test. The results of these analyses demonstrate the effectiveness of our emotion-inducing experimental design, the gender difference in emotional responses, and the coexistence of multiple emotions/expressions. Facial image sequences are more informative than apex images for both expression and emotion recognition. Labeling an expression image or sequence with multiple categories together with their intensities could be a better approach than labeling the expression image or sequence with one dominant category. The results also demonstrate both the importance of facial expressions as a means of communication to convey affective states and the diversity of the displayed ma- ifestations of felt emotions. There are indeed some significant differences between the temperature difference data of most posed and spontaneous facial expressions, many of which are found in the forehead and cheek regions.

Research on the Picture Database of Minority Emotion

The Emotional Recognition and Sharing-Space Applying in University Library Based on the Analysis of Tongue Image

An Image Retrieval System with Color Emotion Query

The pictorial set of Emotional Social Interactive Scenarios between Chinese Adults (ESISCA): Development and validation

An image-based investigation on color associations among 100 Chinese emotion words

Inferring Emotional Tags From Social Images With User Demographics.

Psychological Counseling and Character Analysis Algorithm Based on Image Emotion

Emotional Semantic Detection from Multimedia

Building a Chinese Natural Emotional Audio-Visual Database

An EEG-Based Multi-Modal Emotion Database with Both Posed and Authentic Facial Actions for Emotion Analysis

Image Database System of Clothing Emotion

A Novel Emotion Lexicon for Chinese Emotional Expression Analysis on Weibo: Using Grounded Theory and Semi-Automatic Methods

MPED: A Multi-Modal Physiological Emotion Database for Discrete Emotion Recognition

"Distant" Pictures Benefit Emotion Regulation in Emotion Disclosure on WeChat Moments

HEU Emotion: A Large-scale Database for Multi-modal Emotion Recognition in the Wild

Analyses of a Multimodal Spontaneous Facial Expression Database

Text-based emotion classification using emotion cause extraction

A Chinese Multi-label Affective Computing Dataset Based on Social Media Network Users

Extracting human emotions at different places based on facial expressions and spatial clustering analysis

The Analysis of PAD Emotional State Model Based on Emotion Pictures

Semi-Supervised Classification of Emotional Pictures Based on Feature Combination