Abstract:In the quest for human-robot interaction (HRI), leading to the development of emotion recognition, learning, and analysis capabilities, robotics plays a significant role in human perception, attention, decision-making, and social communication. However, the accurate recognition of emotions in HRI remains a challenge. This is due to the coexistence of multiple sources of information in utilizing multimodal facial expressions and head poses as multiple convolutional neural networks (CNN) and deep learning are combined. This research analyzes and improves the robustness of emotion recognition, and proposes a novel approach that optimizes traditional deep neural networks that fall into poor local optima when optimizing the weightings of the deep neural network using standard methods. The proposed approach adaptively finds the better weightings of the network, resulting in a hybrid genetic algorithm with stochastic gradient descent (HGASGD). This hybrid algorithm combines the inherent, implicit parallelism of the genetic algorithm with the better global optimization of stochastic gradient descent (SGD). An experiment shows the effectiveness of our proposed approach in providing complete emotion recognition through a combination of multimodal data, CNNs, and HGASGD, indicating that it represents a powerful tool in achieving interactions between humans and robotics. To validate and test the effectiveness of our proposed approach through experiments, the performance and reliability of our approach and two variants of HGASGD FER are compared using a large dataset of facial images. Our approach integrates multimodal information from facial expressions and head poses, enabling the system to recognize emotions better. The results show that CNN-HGASGD outperforms CNNs-SGD and other existing state-of-the-art methods in terms of FER.

Leveraging Previous Facial Action Units Knowledge for Emotion Recognition on Faces

Cgan Based Facial Expression Recognition for Human-Robot Interaction

Facial Action Units Recognition: A Comparative Study

Facial Expression Recognition Based on Facial Action Unit

Efficient Facial Expression Recognition with Representation Reinforcement Network and Transfer Self-Training for Human–Machine Interaction

Multimodal Facial Emotion Recognition Using Improved Convolution Neural Networks Model

A Deep Learning Perspective on the Origin of Facial Expressions

SAANet: Siamese Action-Units Attention Network for Improving Dynamic Facial Expression Recognition

Facial Expression Recognition Using Hybrid Features of Pixel and Geometry

3D Facial Action Units Recognition for Emotional Expression

Methods for Facial Expression Recognition with Applications in Challenging Situations

Action unit analysis enhanced facial expression recognition by deep neural network evolution

Compound facial expressions recognition approach using DCGAN and CNN

Facial Action Units Recognition by De-expression Residue Learning

Facial Expressions Recognition for Human-Robot Interaction Using Deep Convolutional Neural Networks with Rectified Adam Optimizer

An Action Unit Co-Occurrence Constraint 3DCNN Based Action Unit Recognition Approach.

A novel Leaky Rectified Triangle Linear Unit based Deep Convolutional Neural Network for facial emotion recognition

Facial Emotion Recognition Based on Brain and Machine Collaborative Intelligence

An Ensemble Approach for Facial Expression Analysis in Video

Video facial emotion recognition based on local enhanced motion history image and CNN-CTSLSTM networks

Facial Emotion Recognition Combining Auxiliary Classifiers and Multiscale CBAM Attention Mechanisms