Abstract:Abstract Objective. Cultures have essential influences on emotions. However, most studies on cultural influences on emotions are in the areas of psychology and neuroscience, while the existing affective models are mostly built with data from the same culture. In this paper, we identify the similarities and differences among Chinese, German, and French individuals in emotion recognition with electroencephalogram (EEG) and eye movements from an affective computing perspective. Approach. Three experimental settings were designed: intraculture subject dependent, intraculture subject independent, and cross-culture subject independent. EEG and eye movements are acquired simultaneously from Chinese, German, and French subjects while watching positive, neutral, and negative movie clips. The affective models for Chinese, German, and French subjects are constructed by using machine learning algorithms. A systematic analysis is performed from four aspects: affective model performance, neural patterns, complementary information from different modalities, and cross-cultural emotion recognition. Main results. From emotion recognition accuracies, we find that EEG and eye movements can adapt to Chinese, German, and French cultural diversities and that a cultural in-group advantage phenomenon does exist in emotion recognition with EEG. From the topomaps of EEG, we find that the γ and β bands exhibit decreasing activities for Chinese, while for German and French, θ and α bands exhibit increasing activities. From confusion matrices and attentional weights, we find that EEG and eye movements have complementary characteristics. From a cross-cultural emotion recognition perspective, we observe that German and French people share more similarities in topographical patterns and attentional weight distributions than Chinese people while the data from Chinese are a good fit for test data but not suitable for training data for the other two cultures. Significance. Our experimental results provide concrete evidence of the in-group advantage phenomenon, cultural influences on emotion recognition, and different neural patterns among Chinese, German, and French individuals.

The Classification of Facial Expressions with Multi-Cultural Backgrounds: an Event-Related Fmri Study

Single Trial Classification for the Categories of Perceived Emotional Facial Expressions: an Event-Related Fmri Study.

Social Judgments from Faces Are Universal

Network Representations of Facial and Bodily Expressions: Evidence from Multivariate Connectivity Pattern Classification.

Facial expression recognition: A meta-analytic review of theoretical models and neuroimaging evidence

The Preliminary Study of Facial Expression Recognition in Normal Adults by Fmri

Deep learning reveals what facial expressions mean to people in different cultures

Neural Substrates for Explicit Recognition of Dynamic Facial Expression by Fmri

Neural Univariate Activity and Multivariate Pattern in the Posterior Superior Temporal Sulcus Differentially Encode Facial Expression and Identity

Facial Expression Decoding Based on Fmri Brain Signal

Decoding facial expressions based on face-selective and motion-sensitive areas

Development of a Facial-Expression Database of Chinese Han, Hui and Tibetan People.

A Study on Neural Mechanism of Face Processing Based on Fmri

Multimodal Emotion Classification Method and Analysis of Brain Functional Connectivity Networks

Identifying similarities and differences in emotion recognition with EEG and eye movements among Chinese, German, and French People

Decoding the temporal representation of facial expression in face-selective regions

The other-race effect in facial expression processing: Behavioral and ERP evidence from a balanced cross-cultural study in women

Cross-Cultural Emotion Recognition with EEG and Eye Movement Signals Based on Multiple Stacked Broad Learning System

Facial Expressions Recognition Based on Cognition and Mapped Binary Patterns.

Survey of the Facial Information Cognition: Identity, Affective and Cross-Culture Analysis.

Study on emotion recognition bias in different regional groups