Abstract:This paper presents an integrated mapping of motion and visualization scheme based on a Mixed Reality (MR) subspace approach for the intuitive and immersive telemanipulation of robotic arm-hand systems. The effectiveness of different control-feedback methods for the teleoperation system is validated and compared. The robotic arm-hand system consists of a 6 Degrees-of-Freedom (DOF) industrial manipulator and a low-cost 2-finger gripper, which can be manipulated in a natural manner by novice users physically distant from the working site. By incorporating MR technology, the user is fully immersed in a virtual operating space augmented by real-time 3D visual feedback from the robot working site. Imitation-based velocity-centric motion mapping is implemented via the MR subspace to accurately track operator hand movements for robot motion control and enables spatial velocity-based control of the robot Tool Center Point (TCP). The user control space and robot working space are overlaid through the MR subspace, and the local user and a digital twin of the remote robot share the same environment in the MR subspace. The MR-based motion and visualization mapping scheme for telerobotics is compared to conventional 2D Baseline and MR tele-control paradigms over two tabletop object manipulation experiments. A user survey of 24 participants was conducted to demonstrate the effectiveness and performance enhancements enabled by the proposed system. The MR-subspace-integrated 3D mapping of motion and visualization scheme reduced the aggregate task completion time by 48% compared to the 2D Baseline module and 29%, compared to the MR SpaceMouse module. The perceived workload decreased by 32% and 22%, compared to the 2D Baseline and MR SpaceMouse approaches.

Design and Development of Teleoperation Interactive System for 7-DOF Space Redundant Manipulator

Enable Intuitive and Immersive Teleoperation: Design, Modeling and Control of a Novel Wearable Exoskeleton

Human-machine Interaction System Design for Space-oriented Manipulator Teleoperation

Teleoperation System of Internet-Based Multi-Operator Multi-Mobile-Manipulator

A Robotic Teleoperation System for Precise Robotic Manipulation by Human-Machine Interaction

Development of a Synchronized Human-Robot-Virtuality Interaction System Using Cooperative Robot and Motion Capture Device

Modular Development of Master-Slave Asymmetric Teleoperation Systems with a Novel Workspace Mapping Algorithm

Design and Development of Teleoperation System for Space Robot

Interface Design of a Human-Robot Interaction System for Dual-Manipulators Teleoperation Based on Virtual Reality *

Research and Implementation of Teleoperation System for Space Flexible Manipulator

Control Scheme and Kinematics Analysis for Space Teleoperation Based on 6-DOF Decoupled Manipulator

A heterogeneous master-slave teleoperation method for 7-DOF manipulator

Teleoperation of a space robot experiment system

Research and Implementation of Teleoperation System Based on Visual-Haptic Aid

Mixed-Reality-Enhanced Human–Robot Interaction with an Imitation-Based Mapping Approach for Intuitive Teleoperation of a Robotic Arm-Hand System

Design and implementation of a dexterous human-robot interaction system based on haptic shared control

The design of multimodal human-machine interface for teleoperation

Development of human-machine interface for teleoperation of a mobile manipulator

Design of a coordinated control strategy for multi-mobile-manipulator cooperative teleoperation system

Human Factors Design and Evaluation of China's Space Manipulator Teleoperation System

A Biomimetic Planning Control Method To Anthropomorphize A 7-Dof Redundant Manipulator In Unstructured Environments