Abstract:Due to the lack of transparent and friendly human–robot interaction (HRI) interface, as well as various uncertainties, it is usually a challenge to remotely manipulate a robot to accomplish a complicated task. To improve the teleoperation performance, we propose a new perception mechanism by integrating a novel learning method to operate the robots in the distance. In order to enhance the perception of the teleoperation system, we utilize a surface electromyogram signal to extract the human operator’s muscle activation. As a response to the changes in the external environment, as sensed through haptic and visual feedback, a human operator naturally reacts with various muscle activations. By imitating the human behaviors in task execution, not only motion trajectory but also arm stiffness adjusted by muscle activation, it is expected that the robot would be able to carry out the repetitive tasks autonomously or uncertain tasks with improved intelligence. To this end, we develop a robot learning algorithm based on probability statistics under an integrated framework of the hidden semi-Markov model (HSMM) and the Gaussian mixture method. This method is employed to obtain a generative task model based on the robot’s trajectory. Then, Gaussian mixture regression based on HSMM is applied to correct the robot trajectory with the reproduced results from the learned task model. The execution procedures consist of a learning phase and a reproduction phase. To guarantee the stability, immersion, and maneuverability of the teleoperation system, a variable gain control method that involves electromyography (EMG) is introduced. Experimental results have demonstrated the effectiveness of the proposed method. Note to Practitioners—This paper is inspired by the limitations of teleoperation to perform a task with unfriendly HRI and lack of intelligence. The human operators need to concentrate on the manipulation in the traditional setup of a teleoperation system; thus, it is quite a labor intensive for a human operator. This is a huge challenge for the requirement of increasingly complicated, diverse tasks in teleoperation. Therefore, efficient ways of the robot intelligence need to be urgently developed for the telerobots. In this paper, we develop a robot intelligence framework by merging robot learning technology and perception mechanism. The proposed framework is effective where the task performed with repeatability and rapidity in a teleoperated mode. The proposed method includes three following ideas: 1) remote operation information can be actively sensed by infusing muscle activation with a haptics EMG perception mechanism; 2) the robot intelligence can be enhanced by employing a robot learning method. The developed approach is verified by the experimental results; and 3) the proposed method can be potentially used for telemanufacturing, teletehabilitation, and telemedicine, and so on. In our future work, more interactive information between humans and telerobots should be taken into consideration in the telerobot perception system to enhance the robot intelligence.

The design of multimodal human-machine interface for teleoperation

Design of multi-modal feedback channel of human–robot cognitive interface for teleoperation in manufacturing

Unified Contact Model and Hybrid Motion/force Control for Teleoperated Manipulation in Unknown Environments

Interface Design of a Human-Robot Interaction System for Dual-Manipulators Teleoperation Based on Virtual Reality *

Teleoperation System of Internet-Based Multi-Operator Multi-Mobile-Manipulator

Development of a Synchronized Human-Robot-Virtuality Interaction System Using Cooperative Robot and Motion Capture Device

Multimodal Interfaces for Effective Teleoperation

An Intelligent Interaction Framework for Teleoperation Based on Human-Machine Cooperation

Human-machine Interface for a Master-Slave Hydraulic Manipulator with Vision Enhancement and Auditory Feedback

A Digital Map Based Mobile Robot Teleoperation System

A Robot Learning Method with Physiological Interface for Teleoperation Systems

Development of human-machine interface for teleoperation of a mobile manipulator

Enhanced teleoperation performance using hybrid control and virtual fixture

Research on Mixed Reality Visual Augmentation Method for Teleoperation Interactive System.

Haptics Electromyogrphy Perception and Learning Enhanced Intelligence for Teleoperated Robot

Robotic Teleoperation Systems Using a Wearable Multimodal Fusion Device

Teleoperation and Visualization Interfaces for Remote Intervention in Space

A Gesture- and Speech-Guided Robot Teleoperation Method Based on Mobile Interaction With Unrestricted Force Feedback

Telepresence augmentation for visual and haptic guided immersive teleoperation of industrial manipulator

A Multi‐Module Sensing and Bi‐Directional HMI Integrating Interaction, Recognition, and Feedback for Intelligent Robots

Teleoperation System for Outdoor Mobile Robot