Abstract:Human feelings expressed through verbal (e.g. voice) and non-verbal communication channels (e.g. face or body) can influence either human actions or interactions. In the literature, most of the attention was given to facial expressions for the analysis of emotions conveyed through non-verbal behaviors. Despite this, psychology highlights that the body is an important indicator of the human affective state in performing daily life activities. Therefore, this paper presents a novel method for affective action and interaction recognition from videos, exploiting multi-view representation learning and only full-body handcrafted characteristics selected following psychological and proxemic studies. Specifically, 2D skeletal data are extracted from RGB video sequences to derive diverse low-level skeleton features, i.e. multi-views, modeled through the bag-of-visual-words clustering approach generating a condition-related codebook. In this way, each affective action and interaction within a video can be represented as a frequency histogram of codewords. During the learning phase, for each affective class, training samples are used to compute its global histogram of codewords stored in a database and later used for the recognition task. In the recognition phase, the video frequency histogram representation is matched against the database of class histograms and classified as the closest affective class in terms of Euclidean distance. The effectiveness of the proposed system is evaluated on a specifically collected dataset containing 6 emotion for both actions and interactions, on which the proposed system obtains 93.64% and 90.83% accuracy, respectively. In addition, the devised strategy also achieves in line performances with other literature works based on deep learning when tested on a public collection containing 6 emotions plus a neutral state, demonstrating the effectiveness of the presented approach and confirming the findings in psychological and proxemic studies.

Affective Actions Recognition in Dyadic Interactions Based on Generative and Discriminative Models

Cgan Based Facial Expression Recognition for Human-Robot Interaction

A Multi-Featured Analysis for Body Sensor Networks-based Affective Actions Recognition.

Toward a Decision-Theoretic Framework for Affect Recognition and User Assistance

Modeling Mutual Influence Of Multimodal Behavior In Affective Dyadic Interactions

Efficient Interaction Recognition Through Positive Action Representation

Human-Robot Emotional Interaction Model Based on Reinforcement Learning

Modeling Dynamics of Expressive Body Gestures in Dyadic Interactions

Generating and Describing Affective Human-Agent Interaction

A Two-Step Shapelets Based Framework for Interactional Activities Recognition

Interaction recognition through positive action representation

Affect Control Processes: Intelligent Affective Interaction using a Partially Observable Markov Decision Process

Affective Behaviour Analysis via Integrating Multi-Modal Knowledge

Affective Interaction Recognition Using Spatio-Temporal Features and Context

Affective Action and Interaction Recognition by Multi-View Representation Learning from Handcrafted Low-Level Skeleton Features

Human Interaction Recognition Framework based on Interacting Body Part Attention

Affect-Driven Learning of Robot Behaviour for Collaborative Human-Robot Interactions

A MultiModal Social Robot Toward Personalized Emotion Interaction

Recognising human interaction from videos by a discriminative model

An Effective Ensemble Learning Framework for Affective Behaviour Analysis

Towards Context-Aware Facial Emotion Reaction Database for Dyadic Interaction Settings