Abstract:The main goal of this research is to develop a highly advanced anthropomorphic control system utilizing multiple sensor technologies to achieve precise control of a robotic arm. Combining Kinect and IMU sensors, together with a data glove, we aim to create a multimodal sensor system for capturing rich information of human upper body movements. Specifically, the four angles of upper limb joints are collected using the Kinect sensor and IMU sensor. In order to improve the accuracy and stability of motion tracking, we use the Kalman filter method to fuse the Kinect and IMU data. In addition, we introduce data glove technology to collect the angle information of the wrist and fingers in seven different directions. The integration and fusion of multiple sensors provides us with full control over the robotic arm, giving it flexibility with 11 degrees of freedom. We successfully achieved a variety of anthropomorphic movements, including shoulder flexion, abduction, rotation, elbow flexion, and fine movements of the wrist and fingers. Most importantly, our experimental results demonstrate that the anthropomorphic control system we developed is highly accurate, real-time, and operable. In summary, the contribution of this study lies in the creation of a multimodal sensor system capable of capturing and precisely controlling human upper limb movements, which provides a solid foundation for the future development of anthropomorphic control technologies. This technology has a wide range of application prospects and can be used for rehabilitation in the medical field, robot collaboration in industrial automation, and immersive experience in virtual reality environments.

Robotic Teleoperation Systems Using a Wearable Multimodal Fusion Device

Enable Intuitive and Immersive Teleoperation: Design, Modeling and Control of a Novel Wearable Exoskeleton

A Sensor Glove Based on Inertial Measurement Unit for Robot Teleoperetion.

An IoT-Enabled Telerobotic-Assisted Healthcare System Based on Inertial Motion Capture.

Sensor Fusion-Based Teleoperation Control of Anthropomorphic Robotic Arm

Whole-Body Teleoperation Control of Dual-Arm Robot Using Sensor Fusion

A Robotic Teleoperation System for Precise Robotic Manipulation by Human-Machine Interaction

Sensor Fusion Based Humanoid Robotic arm Control*

TeleFMG: A Wearable Force-Myography Device for Natural Teleoperation of Multi-finger Robotic Hands

Development of a Synchronized Human-Robot-Virtuality Interaction System Using Cooperative Robot and Motion Capture Device

Sensor Fusion-Based Anthropomorphic Control of a Robotic Arm

A Robotic Hand-Arm Teleoperation System Using Human Arm/Hand with A Novel Data Glove

An Agile Large-Workspace Teleoperation Interface Based on Human Arm Motion and Force Estimation

A Wearable Upper Limb Exoskeleton for Intuitive Teleoperation of Anthropomorphic Manipulators

Skill Learning for Human-Robot Interaction Using Wearable Device

An IMUs and potentiometer-based controller for robotic arm-hand teleoperation

Seamless Robot Teleoperation: Intuitive Control Through Hand Gestures and Neural Network Decoding

A Mobile Robot Hand-Arm Teleoperation System by Vision and IMU

Wearable teleoperation controller with 2-DoF robotic arm and haptic feedback for enhanced interaction in virtual reality

A Multi‐Module Sensing and Bi‐Directional HMI Integrating Interaction, Recognition, and Feedback for Intelligent Robots

An Integrated Vision-Based System for Efficient Robot Arm Teleoperation