Abstract:In the quest for human-robot interaction (HRI), leading to the development of emotion recognition, learning, and analysis capabilities, robotics plays a significant role in human perception, attention, decision-making, and social communication. However, the accurate recognition of emotions in HRI remains a challenge. This is due to the coexistence of multiple sources of information in utilizing multimodal facial expressions and head poses as multiple convolutional neural networks (CNN) and deep learning are combined. This research analyzes and improves the robustness of emotion recognition, and proposes a novel approach that optimizes traditional deep neural networks that fall into poor local optima when optimizing the weightings of the deep neural network using standard methods. The proposed approach adaptively finds the better weightings of the network, resulting in a hybrid genetic algorithm with stochastic gradient descent (HGASGD). This hybrid algorithm combines the inherent, implicit parallelism of the genetic algorithm with the better global optimization of stochastic gradient descent (SGD). An experiment shows the effectiveness of our proposed approach in providing complete emotion recognition through a combination of multimodal data, CNNs, and HGASGD, indicating that it represents a powerful tool in achieving interactions between humans and robotics. To validate and test the effectiveness of our proposed approach through experiments, the performance and reliability of our approach and two variants of HGASGD FER are compared using a large dataset of facial images. Our approach integrates multimodal information from facial expressions and head poses, enabling the system to recognize emotions better. The results show that CNN-HGASGD outperforms CNNs-SGD and other existing state-of-the-art methods in terms of FER.

Hierarchical Co-Consistency Quantization and Information Refining Binary Network for Facial Expression Recognition in Human–Robot Interaction

Facial Expression Recognition Based on Zero-Addition Pretext Training and Feature Conjunction-Selection Network in Human–Robot Interaction

Cgan Based Facial Expression Recognition for Human-Robot Interaction

Efficient Facial Expression Recognition with Representation Reinforcement Network and Transfer Self-Training for Human–Machine Interaction

SDNET: Lightweight Facial Expression Recognition For Sample Disequilibrium.

Combining 2D Gabor and Local Binary Pattern for Facial Expression Recognition Using Extreme Learning Machine

Fine-grained Facial Expression Recognition Via Relational Reasoning and Hierarchical Relation Optimization

Real-time Facial Expression Recognition on Robot for Healthcare

Facial Expression Recognition Via Region-Based Convolutional Fusion Network.

Facial expression recognition with convolutional neural networks via a new face cropping and rotation strategy

A joint learning method with consistency-aware for low-resolution facial expression recognition

Hierarchical Facial Expression Recognition

Real-Time Facial Expression Recognition Based on Attention Mechanism

Enhanced Hybrid Vision Transformer with Multi-Scale Feature Integration and Patch Dropping for Facial Expression Recognition

Facial Expressions Recognition for Human-Robot Interaction Using Deep Convolutional Neural Networks with Rectified Adam Optimizer

Multimodal Facial Emotion Recognition Using Improved Convolution Neural Networks Model

A convolution-transformer dual branch network for head-pose and occlusion facial expression recognition

Facial expression recognition based on deep convolution long short-term memory networks of double-channel weighted mixture

FER Based on the Improved Convex Nonnegative Matrix Factorization Feature

The Facial Expression Recognition Method Based on Image Fusion and CNN

Facial Expression Recognition Using Hybrid Features of Pixel and Geometry