Abstract:Human feelings expressed through verbal (e.g. voice) and non-verbal communication channels (e.g. face or body) can influence either human actions or interactions. In the literature, most of the attention was given to facial expressions for the analysis of emotions conveyed through non-verbal behaviors. Despite this, psychology highlights that the body is an important indicator of the human affective state in performing daily life activities. Therefore, this paper presents a novel method for affective action and interaction recognition from videos, exploiting multi-view representation learning and only full-body handcrafted characteristics selected following psychological and proxemic studies. Specifically, 2D skeletal data are extracted from RGB video sequences to derive diverse low-level skeleton features, i.e. multi-views, modeled through the bag-of-visual-words clustering approach generating a condition-related codebook. In this way, each affective action and interaction within a video can be represented as a frequency histogram of codewords. During the learning phase, for each affective class, training samples are used to compute its global histogram of codewords stored in a database and later used for the recognition task. In the recognition phase, the video frequency histogram representation is matched against the database of class histograms and classified as the closest affective class in terms of Euclidean distance. The effectiveness of the proposed system is evaluated on a specifically collected dataset containing 6 emotion for both actions and interactions, on which the proposed system obtains 93.64% and 90.83% accuracy, respectively. In addition, the devised strategy also achieves in line performances with other literature works based on deep learning when tested on a public collection containing 6 emotions plus a neutral state, demonstrating the effectiveness of the presented approach and confirming the findings in psychological and proxemic studies.

Emotion pattern detection on facial videos using functional statistics

Multi-modal emotion analysis from facial expressions and electroencephalogram.

Multimodal Emotion Recognition by Combining Physiological Signals and Facial Expressions: a Preliminary Study.

Real-time Automatic Emotion Recognition from Body Gestures

Detection of (Hidden) Emotions from Videos using Muscles Movements and Face Manifold Embedding

Recognizing Emotions Based on Human Actions in Videos.

An Ensemble Approach for Facial Expression Analysis in Video

Sensing emotional valence and arousal dynamics through automated facial action unit analysis

Automatic Facial Expression Recognition in Standardized and Non-standardized Emotional Expressions

Automatic Recognition of Facial Displays of Unfelt Emotions

AN ALGORITHM FOR DETECTING NON-VERBAL MARKERS OF HUMAN BEHAVIOR ON VIDEO

Affective Action and Interaction Recognition by Multi-View Representation Learning from Handcrafted Low-Level Skeleton Features

Human emotion recognition by analyzing facial expressions, heart rate and blogs using deep learning method

A method for recognizing facial expression intensity based on facial muscle variations

Emotion recognition through facial expression analysis based on a neurofuzzy network

Human Face Emotion in 3D Using Machine Learning

Non-contact Emotion Analysis Based on Videos.

Facial Action Units Recognition: A Comparative Study

Happy Index - Analysis Based on Automatic Recognition of Emotion Flow.

Facial Emotions Recognition Using Deep Learning Technology

A Comparison of the Affectiva iMotions Facial Expression Analysis Software With EMG for Identifying Facial Expressions of Emotion