Abstract:This paper designed a voice interactive robot system that can conveniently execute assigned service tasks in real-life scenarios. It is equipped without a microphone where users can control the robot with spoken commands; the voice commands are then recognized by a well-trained deep neural network model of automatic speech recognition (ASR), which enables the robot to execute and complete the command based on the navigation of a real-time simultaneous localization and mapping (SLAM) algorithm. The voice interaction recognition model is divided into two parts: (1) speaker separation and (2) ASR. The speaker separation is applied by a deep-learning system consisting of eight convolution layers, one LSTM layer, and two fully connected (FC) layers to separate the speaker’s voice. This model recognizes the speaker’s voice as a referrer that separates and holds the required voiceprint and removes noises from other people’s voiceprints. Its automatic speech recognition uses the novel sandwich-type conformer model with a stack of three layers, and combines convolution and self-attention to capture short-term and long-term interactions. Specifically, it contains a multi-head self-attention module to directly convert the voice data into text for command realization. The RGB-D vision-based camera uses a real-time appearance-based mapping algorithm to create the environment map and replace the localization with a visional odometer to allow the robot to navigate itself. Finally, the proposed ASR model was tested to check if the desired results will be obtained. Performance analysis was applied to determine the robot’s environment isolation and voice recognition abilities. The results showed that the practical robot system successfully completed the interactive service tasks in a real environment. This experiment demonstrates the outstanding performance with other ASR methods and voice control mobile robot systems. It also verified that the designed voice interaction recognition system enables the mobile robot to execute tasks in real-time, showing that it is a convenient way to complete the assigned service applications.

Research of voice synthesis in humanoid robot

Giving Robots a Voice: Human-in-the-Loop Voice Creation and open-ended Labeling

Robot‐by‐voice: experiments on commanding an industrial robot using the human voice

Research on Intelligent-Robot-Oriented Speech Interactive Technology and Module

Vocal emotion of humanoid robots: a study from brain mechanism.

Robot voice a voice controlled robot using arduino

Voice Interaction Recognition Design in Real-Life Scenario Mobile Robot Applications

Make-A-Voice: Unified Voice Synthesis With Discrete Representation

Contactless Interaction System Based on Facial Expression Recognition for Humanoid Piano Robot

Robotic voice assistant equipped with binaural audio

New Multi-modeling Voice Interactive System

No More Mumbles: Enhancing Robot Intelligibility through Speech Adaptation

Scenario-Based Programming of Voice-Controlled Medical Robotic Systems

Voice frequency impact for voice control of collaborative robots

Voice Synthesis Improvement by Machine Learning of Natural Prosody

A Voice Control Platform of Mobile Robot through ROS

Body of Her: A Preliminary Study on End-to-End Humanoid Agent

Spoken language interaction with robots: Recommendations for future research

Facilitating Human-Robot Collaboration through Natural Vocal Conversations

Voice control interface for surgical robot assistants

Analysis of voice control of a collaborative robot