Abstract:A prosthetic hand is one of the main ways to help patients with upper limb disabilities regain their daily living abilities. Prosthetic hand manipulation must be coordinated with the user's action intention. Therefore, the key to the control of the prosthetic hand is to recognize the action intention of the upper limb. At present, there are still problems such as difficulty in decoding information and a low recognition rate of identifying action intention with EMG signals and EEG signals. While inertial sensors have the advantages of low cost and high accuracy and posture information can characterize the upper limb motion state, visual information has the advantages of high information and being able to detect the type of target objects, which can be complementarily fused with inertial sensors to further grasp the human motion requirements. Therefore, this paper proposes an upper limb action intention recognition method based on the fusion of posture information and visual information. The inertial sensor is used to collect the attitude angle data during the movement of the upper limb, and according to the similarity of the human upper limb structure to the linkage mechanism, a model of the upper limb of the human body is established using the positive kinematics theory of a mechanical arm to solve the upper limb end positions. The upper limb end positions were classified into three categories: torso front, upper body nearby, and the initial position, and a multilayer perceptron model was trained to learn the classification relationships. In addition, a miniature camera was installed on the hand to obtain visual image information during upper limb movement. The target objects are detected using the YOLOv5 deep learning method, and then, the target objects are classified into two categories: wearable items and non-wearable items. Finally, the upper limb intention is jointly decided by the upper limb motion state, target object type, and upper limb end position to achieve the control of the prosthetic hand. We applied the upper limb intention recognition method to the experimental system of a mechanical prosthetic hand and invited several volunteers to test it. The experimental results showed that the intention recognition success rate reached 92.4%, which verifies the feasibility and practicality of the upper limb action intention recognition method based on the fusion of posture information and visual information.

Study of Muscle Redistribution Technique in Signal Recognition of Intelligent Bionic Hand

Application of muscle redistribution in the signal recognition of an intelligent bionic prosthesis

Anatomical study on the application of muscle redistribution technology in intelligent bionic hands

Muscle Redistribution Surgery Based Capacitive Sensing for Upper-Limb Motion Recognition: Preliminary Results.

A Novel Interface Device Developed Based on MRT for Prosthetic Hand

Training Wrist Extensor Function and Detecting Unwanted Movement Strategies in an EMG-controlled Visuomotor Task.

Improving sEMG-based motion intention recognition for upper-limb amputees using transfer learning

Experimental Study of an Emg-Controlled 5-Dof Anthropomorphic Prosthetic Hand for Motion Restoration

Myoelectric prosthesis hand grasp control following targeted muscle reinnervation in individuals with transradial amputation

Two Ways to Improve Myoelectric Control for a Transhumeral Amputee after Targeted Muscle Reinnervation: a Case Study

Muscle Redistribution Technique for Expressing Motion Intention in Patients with Wrist-Level Amputation

Improving Motion Intention Recognition for Trans-Radial Amputees Based on Semg and Transfer Learning

Research on Upper Limb Action Intention Recognition Method Based on Fusion of Posture Information and Visual Information

A Wearable and Portable Real-Time Control Gesture Recognition System for Bionic Manipulator

[Targeted muscle reinnervation: a surgical technique of human-machine interface for intelligent prosthesis]

A Novel Hybrid Approach to Pattern Recognition of Finger Movements and Grasping Gestures in Upper Limb Amputees

Decoding a new neural machine interface for control of artificial limbs

Biomechanical Signals Based Pattern Recognition Using a Developed Algorithm in Upper Limb Amputees

A motion recognition model for upper-limb rehabilitation exercises

Clinical implementation of a bionic hand controlled with kineticomyographic signals

Recognition of Upper Limb Action Intention Based on IMU