Abstract:With the increasing demands on human–machine interaction in intelligent neural rehabilitation, surface electromyogram (sEMG) decoders enabling hand gesture recognition have attracted more attentions. Although previous studies have proposed diverse models, their low inter-subject and inter-day robustness still cannot satisfy the demands in real world applications. In this work, we employed deep transfer learning to adapt the mismatch between source and target domains for robust hand gesture recognition. We systematically investigated 5 deep transfer learning algorithms on end-to-end convolutional neural network (CNN) architecture. Statistical machine learning methods with benchmark handcrafted features and general domain adaptation algorithms were also implemented as the strong baseline performance for the comparisons. The performance of the model under both inter-subject and inter-day applications was evaluated using high-density sEMG (HDsEMG) signals with 256 channels acquired from 36 subjects twice on 2 days in various rigorous validation protocols. Importantly, we examined why the deep transfer learning functioned by monitoring both the intra-domain and inter-domain mismatch between the learned features during model training. The results demonstrated that, without deep transfer learning, the intra-domain mismatch between samples under the same gestures decreased but the mismatch between domains (source and target domains) increased rapidly. The clustering of samples under the same gesture came at the expense of higher mismatch between different domains. With deep transfer learning applied, the inter-day and inter-subject performance of CNN was greatly improved (improved from 85.4% to 91.39%, and from 82.22% to 91.38%, respectively), which also outperformed benchmark statistical machine learning models with domain adaptation. In addition, the network structure of the proposed method is simple and can be easily implemented online. Overall, the findings in this work show great promise of transfer learning in hand gesture recognition under inter-subject and inter-day settings, significantly reducing the inconvenience of model re-calibration in practical use.

Post-stroke hand gesture recognition via one-shot transfer learning using prototypical networks

Empowering Hand Rehabilitation with AI-Powered Gesture Recognition: A Study of an sEMG-Based System

A fully connected deep learning approach to upper limb gesture recognition in a secure FES rehabilitation environment

Gesture Classification in Electromyography Signals for Real-Time Prosthetic Hand Control Using a Convolutional Neural Network-Enhanced Channel Attention Model

Activities of Daily Living-Based Rehabilitation System for Arm and Hand Motor Function Retraining After Stroke

A Home-Based Functional Hand-Extremity Assessment System for Stroke Rehabilitation

BiGRA: A Preliminary Bilateral Hand Grip Coordination Rehabilitation Using Home-Based Evaluation System for Stroke Patients

An IoT-Enabled Stroke Rehabilitation System Based on Smart Wearable Armband and Machine Learning

An Unobtrusive Upper-Limb Activity Recognition System Based on Deep Neural Network Fusion for Stroke Survivors.

Proposal of a Wearable Multimodal Sensing-Based Serious Games Approach for Hand Movement Training after Stroke.

A Control and Posture Recognition Strategy for Upper-Limb Rehabilitation of Stroke Patients

Generalizability of Hand-Object Interaction Detection in Egocentric Video across Populations with Hand Impairment

Soft Exoskeleton Glove for Hand Assistance Based on Human-machine Interaction and Machine Learning

sEMG-Based Gesture Classifier for a Rehabilitation Glove

A Wearable Hand Rehabilitation System With Soft Gloves

Deep end-to-end transfer learning for robust inter-subject and inter-day hand gesture recognition using surface EMG

A Probabilistic Modeling Approach to One-Shot Gesture Recognition

Wrist-Worn Hand Gesture Recognition While Walking Via Transfer Learning.

Hand Gesture Recognition based on Surface Electromyography using Convolutional Neural Network with Transfer Learning Method

An Actively Vision-Assisted Low-Load Wearable Hand Function Mirror Rehabilitation System

Wearable Multimodal - Serious Game System For Hand and Cognitive Rehabilitation After Stroke