Abstract:Artificial intelligence provides new feasibilities to the control of dexterous prostheses. To achieve suitable grasps over various objects, a novel computer vision-based classification method assorting objects into different grasp patterns is proposed in this paper. This method can be applied in the autonomous control of the multi-fingered prosthetic hand, as it can help users rapidly complete “reach-and-pick up” tasks on various daily objects with low demand on the myoelectric control. Firstly, an RGB-D image database (121 objects) was established according to four important grasp patterns (cylindrical, spherical, tripod, and lateral). The image samples in the RGB-D dataset were acquired on a large variety of daily objects of different sizes, shapes, postures (16), as well as different illumination conditions (4) and camera positions (4). Then, different inputs and structures of the discrimination model (multilayer CNN) were tested in terms of the classification success rate through cross-validation. Our results showed that depth data play an important role in grasp pattern recognition. The bimodal data (Gray-D) integrating both grayscale and depth information about the objects can improve the classification accuracy acquired from the RGB images (> 10%) effectively. Within the database, the network could achieve the classification with high accuracy (98%); it also has a strong generalization capability on novel samples (93.9 ± 3.0%). We finally applied the method on a dexterous prosthetic hand and tested the whole system on performing the “reach-and-pick up” tasks. The experiments showed that the proposed computer vision-based myoelectric control method (Vision-EMG) could significantly improve the control effectiveness (6.4 s), with comparison to the traditional coding-based myoelectric control method (Coding-EMG, 13 s).

Control of Virtual Prosthesis Based on Mechanomyogram Signal

Improving the Robustness of Human-Machine Interactive Control for Myoelectric Prosthetic Hand During Arm Position Changing

Hand-motion patterns recognition based on mechanomyographic signal analysis

Classification of the Mechanomyogram Signal Using a Wavelet Packet Transform and Singular Value Decomposition for Multifunction Prosthesis Control

EMG control of a 3-DOF anthropomorphic prosthetic hand

Upper-limb prosthetic control using wearable multichannel mechanomyography

Comparison of Electromyography and Mechanomyogram in Control of Prosthetic System in Multiple Limb Positions

Mechanomyography signals processing method using multivariate variational mode decomposition

Experimental Study of an Emg-Controlled 5-Dof Anthropomorphic Prosthetic Hand for Motion Restoration

EMG Pattern Recognition and Grasping Force Estimation: Improvement to the Myocontrol of Multi-Dof Prosthetic Hands.

Control of myoelectric prosthetic hand with a novel proximity-tactile sensor

Human-Machine Interaction Control Based on Force Myograph and Electrical Stimulation Sensory Feedback for Multi-DOF Robotic Hand

A Bio-Mechanical Designed Prosthetic Hand With Multi-Control Strategies

An Integrated Virtual Hand Platform for Evaluation of Model-Based Control of Hand Prosthesis.

An MMG-based Control Method of Prosthetic Manipulators Using Acceleration Sensors

Online recognition of hand EMG patterns and grasping force based on support vector machines

Biomimetic Tactile Data Driven Closed-loop Control of Myoelectric Prosthetic Hand-.

ProRuka: A highly efficient HMI algorithm for controlling a novel prosthetic hand with 6-DOF using sonomyography

Machine learning for hand pose classification from phasic and tonic EMG signals during bimanual activities in virtual reality

Surface Emg For Multi-Pattern Recognition With Sensory Feedback Controller Of Hand Prosthesis System

Computer Vision-Based Grasp Pattern Recognition with Application to Myoelectric Control of Dexterous Hand Prosthesis