Abstract:In the quest for human-robot interaction (HRI), leading to the development of emotion recognition, learning, and analysis capabilities, robotics plays a significant role in human perception, attention, decision-making, and social communication. However, the accurate recognition of emotions in HRI remains a challenge. This is due to the coexistence of multiple sources of information in utilizing multimodal facial expressions and head poses as multiple convolutional neural networks (CNN) and deep learning are combined. This research analyzes and improves the robustness of emotion recognition, and proposes a novel approach that optimizes traditional deep neural networks that fall into poor local optima when optimizing the weightings of the deep neural network using standard methods. The proposed approach adaptively finds the better weightings of the network, resulting in a hybrid genetic algorithm with stochastic gradient descent (HGASGD). This hybrid algorithm combines the inherent, implicit parallelism of the genetic algorithm with the better global optimization of stochastic gradient descent (SGD). An experiment shows the effectiveness of our proposed approach in providing complete emotion recognition through a combination of multimodal data, CNNs, and HGASGD, indicating that it represents a powerful tool in achieving interactions between humans and robotics. To validate and test the effectiveness of our proposed approach through experiments, the performance and reliability of our approach and two variants of HGASGD FER are compared using a large dataset of facial images. Our approach integrates multimodal information from facial expressions and head poses, enabling the system to recognize emotions better. The results show that CNN-HGASGD outperforms CNNs-SGD and other existing state-of-the-art methods in terms of FER.

Speech Fusion Based on Rough Neural Network for Humanoid Intelligent Robots

Cgan Based Facial Expression Recognition for Human-Robot Interaction

Multisensor Data Fusion for Robot Parts Recognition

Neuro-Fuzzy Network Based on Rough Sets and Its Applications

Hierarchical Co-Consistency Quantization and Information Refining Binary Network for Facial Expression Recognition in Human–Robot Interaction

Multi-robot behavior adaptation to local and global communication atmosphere in humans-robots interaction

Realization of the Fusion of the Numerical and Linguistic Information Using a Fuzzy Neural Network

Multi-Robot Behavior Adaptation to Humans' Intention in Human-Robot Interaction Using Information-Driven Fuzzy Friend-Q Learning.

Emotion-Age-Gender-Nationality Based Intention Understanding in Human–Robot Interaction Using Two-Layer Fuzzy Support Vector Regression

Emotional States Based 3-D Fuzzy Atmosfield for Casual Communication Between Humans and Robots

Speech Fusion to Face: Bridging the Gap Between Human's Vocal Characteristics and Facial Imaging

Robust Audio-Visual Speech Recognition Based on Hybrid Fusion

Hybrid Recurrent Neural Network Architecture-Based Intention Recognition for Human-Robot Collaboration

Energy Efficient Graph-Based Hybrid Learning for Speech Emotion Recognition on Humanoid Robot

Multi-characters Fusion Recognition System Based on Neural Network

Multimodal information fusion for human-robot interaction

Fusion of Embeddings Networks for Robust Combination of Text Dependent and Independent Speaker Recognition

Multimodal fusion-powered English speaking robot

No More Mumbles: Enhancing Robot Intelligibility through Speech Adaptation

A Fusion Approach to Spoken Language Identification Based on Combining Multiple Phone Recognizers and Speech Attribute Detectors

Multimodal Facial Emotion Recognition Using Improved Convolution Neural Networks Model