Abstract:Human gait is an emerging biometric feature and contains important information for long-distance emotion recognition. However, sadness and neutral emotions are easily misjudged during the recognition process, because the body posture of the two emotions is quite similar. Existing methods have difficulty in distinguishing these two emotions satisfactorily since they treat all action features equally without differentiating their contribution to recognition, or they ignore the motion information in gait that can express emotion. In this paper, we propose a novel hierarchical attention neural network, which can automatically learn the affective features contained in human motion and action, and effectively distinguish between sad and neutral emotions. The network consists of three modules: motion sentiment module (MSM), action sentiment module (ASM) and emotion classifier. Specifically, MSM is composed of a position encoder and a velocity encoder. It extracts affective features from motion information and helps to distinguish the sad and neutral emotions based on gait velocity. ASM consists of an action encoder that compresses the discriminative human actions into a latent space. Emotion classifier recognizes the emotion based on the outputs from MSM and ASM. Moreover, we present a feature preprocessing method to deal with the problem of imbalanced data categories. Experiments demonstrate that our approach enhances the discriminability of sad and neutral emotions and performs better than many state-of-the-art methods. In addition, ablation experiments further verify that both velocity and action features are important for the gait-based emotion recognition task.

STEP: Spatial Temporal Graph Convolutional Networks for Emotion Perception from Gaits

G-GCSN: Global Graph Convolution Shrinkage Network for Emotion Perception from Gait

A Multi-Head Pseudo Nodes Based Spatial–temporal Graph Convolutional Network for Emotion Perception from GAIT

Take an Emotion Walk: Perceiving Emotions from Gaits Using Hierarchical Attention Pooling and Affective Mapping

Identifying Emotions from Walking using Affective and Deep Features

TT-GCN: Temporal-Tightly Graph Convolutional Network for Emotion Recognition from Gaits

AST-GCN: Augmented Spatial Temporal Graph Convolutional Neural Network for Gait Emotion Recognition

Gait-based Emotion Recognition Using Spatial Temporal Graph Convolutional Networks

Looking into Gait for Perceiving Emotions via Bilateral Posture and Movement Graph Convolutional Networks

Attention-based Temporal Graph Representation Learning for EEG-based Emotion Recognition

ST-Gait++: Leveraging spatio-temporal convolutions for gait-based emotion recognition on videos

Hierarchical-Attention-Based Neural Network for Gait Emotion Recognition

Two Feature Fusion Network Based on Efficient Deep Residual Shrinkage Spatial-Temporal Graph Convolution for Emotion Recognition from Gaits

EPIC: Emotion Perception by Spatio-Temporal Interaction Context of Gait

Generating Emotive Gaits for Virtual Agents Using Affect-Based Autoregression

Multi-task Learning for Gait-Based Identity Recognition and Emotion Recognition Using Attention Enhanced Temporal Graph Convolutional Network.

Data Augmentation by Separating Identity and Emotion Representations for Emotional Gait Recognition.

Automatic Emotion Recognition Based on Non-Contact Gaits Information

EEG-Based Emotion Recognition Using Spatial-Temporal Graph Convolutional LSTM with Attention Mechanism.

EGST: An Efficient Solution for Human Gaits Recognition Using Neuromorphic Vision Sensor

TNTC: Two-Stream Network with Transformer-Based Complementarity for Gait-Based Emotion Recognition