Abstract:A prosthetic hand is one of the main ways to help patients with upper limb disabilities regain their daily living abilities. Prosthetic hand manipulation must be coordinated with the user's action intention. Therefore, the key to the control of the prosthetic hand is to recognize the action intention of the upper limb. At present, there are still problems such as difficulty in decoding information and a low recognition rate of identifying action intention with EMG signals and EEG signals. While inertial sensors have the advantages of low cost and high accuracy and posture information can characterize the upper limb motion state, visual information has the advantages of high information and being able to detect the type of target objects, which can be complementarily fused with inertial sensors to further grasp the human motion requirements. Therefore, this paper proposes an upper limb action intention recognition method based on the fusion of posture information and visual information. The inertial sensor is used to collect the attitude angle data during the movement of the upper limb, and according to the similarity of the human upper limb structure to the linkage mechanism, a model of the upper limb of the human body is established using the positive kinematics theory of a mechanical arm to solve the upper limb end positions. The upper limb end positions were classified into three categories: torso front, upper body nearby, and the initial position, and a multilayer perceptron model was trained to learn the classification relationships. In addition, a miniature camera was installed on the hand to obtain visual image information during upper limb movement. The target objects are detected using the YOLOv5 deep learning method, and then, the target objects are classified into two categories: wearable items and non-wearable items. Finally, the upper limb intention is jointly decided by the upper limb motion state, target object type, and upper limb end position to achieve the control of the prosthetic hand. We applied the upper limb intention recognition method to the experimental system of a mechanical prosthetic hand and invited several volunteers to test it. The experimental results showed that the intention recognition success rate reached 92.4%, which verifies the feasibility and practicality of the upper limb action intention recognition method based on the fusion of posture information and visual information.

Muscle Redistribution Surgery Based Capacitive Sensing for Upper-Limb Motion Recognition: Preliminary Results.

A Preliminary Study of Upper-Limb Motion Recognition with Noncontact Capacitive Sensing.

Forearm Motion Recognition with Noncontact Capacitive Sensing.

Study of Muscle Redistribution Technique in Signal Recognition of Intelligent Bionic Hand

Application of muscle redistribution in the signal recognition of an intelligent bionic prosthesis

Noncontact Capacitive Sensing Based Human Motion Intent Recognition

Capacitive Sensing Based Recognition of Ankle Movement Imagery in Patients after Amputation Surgery

Identification of Muscle Morphology with Noncontact Capacitive Sensing: Preliminary Study.

Non-Periodic Lower-Limb Motion Recognition with Noncontact Capacitive Sensing

Training Wrist Extensor Function and Detecting Unwanted Movement Strategies in an EMG-controlled Visuomotor Task.

Research on Upper Limb Action Intention Recognition Method Based on Fusion of Posture Information and Visual Information

Improving sEMG-based motion intention recognition for upper-limb amputees using transfer learning

Improving the Robustness of Human-Machine Interactive Control for Myoelectric Prosthetic Hand During Arm Position Changing

Improving Motion Intention Recognition for Trans-Radial Amputees Based on Semg and Transfer Learning

Muscle Redistribution Technique for Expressing Motion Intention in Patients with Wrist-Level Amputation

Improved Transfer Learning for Detecting Upper-Limb Movement Intention Using Mechanical Sensors in an Exoskeletal Rehabilitation System

A Noncontact Capacitive Sensing System for Recognizing Locomotion Modes of Transtibial Amputees

Non-contact capacitance sensing for continuous locomotion mode recognition: design specifications and experiments with an amputee

Locomotion mode classification using a wearable capacitive sensing system.

A Novel Motion Recognition Method Based on Force Myography of Dynamic Muscle Contractions

A Wearable Capacitive Sensing System with Phase-Dependent Classifier for Locomotion Mode Recognition