Abstract:Motor disorder of upper limbs has seriously affected the daily life of the patients with hemiplegia after stroke. We developed a wearable supernumerary robotic limb (SRL) system using a hybrid control approach based on motor imagery (MI) and object detection for upper-limb motion assistance. SRL system included an SRL hardware subsystem and a hybrid control software subsystem. The system obtained the patient's motion intention through MI electroencephalogram (EEG) recognition method based on graph convolutional network (GCN) and gated recurrent unit network (GRU) to control the left and right movements of SRL, and the object detection technology was used together for a quick grasp of target objects to compensate for the disadvantages when using MI EEG alone like fewer control instructions and lower control efficiency. Offline training experiment was designed to obtain subjects' MI recognition models and evaluate the feasibility of the MI EEG recognition method; online control experiment was designed to verify the effectiveness of our wearable SRL system. The results showed that the proposed MI EEG recognition method (GCN+GRU) could effectively improve the MI classification accuracy (90.04% %) compared with traditional methods; all subjects were able to complete the target object grasping tasks within 23 seconds by controlling the SRL, and the highest average grasping success rate achieved 90.67% in bag grasping task. The SRL system can effectively assist people with upper-limb motor disorder to perform upper-limb tasks in daily life by natural human-robot interaction, and improve their ability of self-help and enhance their confidence of life.

Learning-Based Multimodal Control for a Supernumerary Robotic System in Human-Robot Collaborative Sorting

Liberating Humanity from Heavy Labor: Dual-Arm Coordination and Teleoperation Control for an Assistive Robot

Wearable Supernumerary Robotic Limb System Using a Hybrid Control Approach Based on Motor Imagery and Object Detection

Compliant Human-Robot Object Transfer Based on Modular 3-Axis Force Sensor for Collaborative Manufacturing.

Multifingered Grasping Based on Multimodal Reinforcement Learning

Multifingered Robot Hand Compliant Manipulation Based on Vision-Based Demonstration and Adaptive Force Control

Exploiting Intrinsic Kinematic Null Space for Supernumerary Robotic Limbs Control

ManipLLM: Embodied Multimodal Large Language Model for Object-Centric Robotic Manipulation

Deep learning-based control framework for dynamic contact processes in humanoid grasping

The Multi-fingered Kinematic Model for Dual-arm Manipulation

Hybrid Robotic Grasping with a Soft Multimodal Gripper and a Deep Multistage Learning Scheme

Hybrid Learning- and Model-Based Planning and Control of In-Hand Manipulation

Multi-Modal Fusion in Contact-Rich Precise Tasks via Hierarchical Policy Learning

Learning compliant grasping and manipulation by teleoperation with adaptive force control

An Exploratory Multi-Session Study of Learning High-Dimensional Body-Machine Interfacing for Assistive Robot Control

“Follower” to “Collaborator”: A robot proactive collaborative controller based on human multimodal information for 3D handling/assembly scenarios

A Human Motion Compensation Framework for a Supernumerary Robotic Arm

Sensing and Control of a Multi-Joint Soft Wearable Robot for Upper-Limb Assistance and Rehabilitation

Unified Human-Robot-Environment Interaction Control in Contact-Rich Collaborative Manipulation Tasks Via Model-Based Reinforcement Learning

Nonprehensile Planar Manipulation through Reinforcement Learning with Multimodal Categorical Exploration

Human-Manipulator Interface Using Hybrid Sensors Via CMAC for Dual Robots.