Abstract:A prosthetic hand is one of the main ways to help patients with upper limb disabilities regain their daily living abilities. Prosthetic hand manipulation must be coordinated with the user's action intention. Therefore, the key to the control of the prosthetic hand is to recognize the action intention of the upper limb. At present, there are still problems such as difficulty in decoding information and a low recognition rate of identifying action intention with EMG signals and EEG signals. While inertial sensors have the advantages of low cost and high accuracy and posture information can characterize the upper limb motion state, visual information has the advantages of high information and being able to detect the type of target objects, which can be complementarily fused with inertial sensors to further grasp the human motion requirements. Therefore, this paper proposes an upper limb action intention recognition method based on the fusion of posture information and visual information. The inertial sensor is used to collect the attitude angle data during the movement of the upper limb, and according to the similarity of the human upper limb structure to the linkage mechanism, a model of the upper limb of the human body is established using the positive kinematics theory of a mechanical arm to solve the upper limb end positions. The upper limb end positions were classified into three categories: torso front, upper body nearby, and the initial position, and a multilayer perceptron model was trained to learn the classification relationships. In addition, a miniature camera was installed on the hand to obtain visual image information during upper limb movement. The target objects are detected using the YOLOv5 deep learning method, and then, the target objects are classified into two categories: wearable items and non-wearable items. Finally, the upper limb intention is jointly decided by the upper limb motion state, target object type, and upper limb end position to achieve the control of the prosthetic hand. We applied the upper limb intention recognition method to the experimental system of a mechanical prosthetic hand and invited several volunteers to test it. The experimental results showed that the intention recognition success rate reached 92.4%, which verifies the feasibility and practicality of the upper limb action intention recognition method based on the fusion of posture information and visual information.

A Hierarchical Hand Motions Recognition Method Based On Imu And Semg Sensors

A Multi-Gestures Recognition System Based on Less Semg Sensors

A Novel Method for Hand Movement Recognition Based on Wavelet Packet Transform and Principal Component Analysis with Surface Electromyogram

A Study On Semg Signals Pattern Recognition Of Key Hand Motions

User-Tailored Hand Gesture Recognition System for Wearable Prosthesis and Armband Based on Surface Electromyogram

Measuring Motor Unit Discharge, Myofiber Vibration, and Haemodynamics for Enhanced Myoelectric Gesture Recognition.

The system of sEMG recognition for prosthetic hand control

Semg-Based Identification of Hand Motion Commands Using Wavelet Neural Network Combined with Discrete Wavelet Transform.

Human Motion Understanding Based On Sensor Fusion

Hand Motion Recognition Based On Pressure Distribution Maps And Ls-Svm

Semg-Based Hand Motion Recognition System Using RMSR and AR Model

Classification of hand movements using variational mode decomposition and composite permutation entropy index with surface electromyogram signals

Hierarchical Classification of Grasp Motions using EMG signals

A Hierarchical Classification of Gestures under Two Force Levels Based on Muscle Synergy.

Research on Upper Limb Action Intention Recognition Method Based on Fusion of Posture Information and Visual Information

Enhanced Performance for Multi-Forearm Movement Decoding Using Hybrid IMU–sEMG Interface

Hand Motions Recognition Based on Semg Nonlinear Feature and Time Domain Feature Fusion

Human Hand Movement Recognition Based on HMM with Hyperparameters Optimized by Maximum Mutual Information

A New Approach for Hand Gesture Recognition Based on the Fusion of sEMG and Impedance Information

Investigation of an Semg Signal-Based Method for Online Hand Movement Classification

Decoding Discrete Gestures Across Different Arm Positions Based on Multimodal Fusion Strategy.