Abstract:This paper designed a voice interactive robot system that can conveniently execute assigned service tasks in real-life scenarios. It is equipped without a microphone where users can control the robot with spoken commands; the voice commands are then recognized by a well-trained deep neural network model of automatic speech recognition (ASR), which enables the robot to execute and complete the command based on the navigation of a real-time simultaneous localization and mapping (SLAM) algorithm. The voice interaction recognition model is divided into two parts: (1) speaker separation and (2) ASR. The speaker separation is applied by a deep-learning system consisting of eight convolution layers, one LSTM layer, and two fully connected (FC) layers to separate the speaker’s voice. This model recognizes the speaker’s voice as a referrer that separates and holds the required voiceprint and removes noises from other people’s voiceprints. Its automatic speech recognition uses the novel sandwich-type conformer model with a stack of three layers, and combines convolution and self-attention to capture short-term and long-term interactions. Specifically, it contains a multi-head self-attention module to directly convert the voice data into text for command realization. The RGB-D vision-based camera uses a real-time appearance-based mapping algorithm to create the environment map and replace the localization with a visional odometer to allow the robot to navigate itself. Finally, the proposed ASR model was tested to check if the desired results will be obtained. Performance analysis was applied to determine the robot’s environment isolation and voice recognition abilities. The results showed that the practical robot system successfully completed the interactive service tasks in a real environment. This experiment demonstrates the outstanding performance with other ASR methods and voice control mobile robot systems. It also verified that the designed voice interaction recognition system enables the mobile robot to execute tasks in real-time, showing that it is a convenient way to complete the assigned service applications.

Research on Speech Control Model of Intelligent Robot Dog

Design of a speech interaction system for man-machine confrontation

Research on Intelligent-Robot-Oriented Speech Interactive Technology and Module

Design of intelligent robot control system based on human–computer interaction

Voice Interaction Recognition Design in Real-Life Scenario Mobile Robot Applications

Remote Control System of Spherical Robot Based on Silent Speech Recognition

Human-robot-interaction using cloud-based speech recognition systems

New Multi-modeling Voice Interactive System

Cross-modal Task Understanding and Execution of Voice-fingertip Reading Instruction by Using Small Family Service Robotic

Decision-Making in Robotic Grasping with Large Language Models.

A Voice Recognition Sensor and Voice Control System in an Intelligent Toy Robot System

Design and Construction of Intelligent Voice Control System

A New Speech Recognition Model in a Human-Robot Interaction Scenario Using NAO Robot: Proposal and Preliminary Model

Speech control based on command matching in vehicle

Vision-aided Speech Recognition System for a Small Four-Legged Robot

Design and Implementation of the Voice Command Recognition and the Sound Source Localization System for Human–Robot Interaction

Design and Realization of an Elderly Escort Robot System

A Selection Method of Speech Vocabulary for Human-Robot Speech Interaction

Quadrupedal Robotic Guide Dog with Vocal Human-Robot Interaction

Spoken language interaction with robots: Recommendations for future research