Abstract:With the increasing demands on human–machine interaction in intelligent neural rehabilitation, surface electromyogram (sEMG) decoders enabling hand gesture recognition have attracted more attentions. Although previous studies have proposed diverse models, their low inter-subject and inter-day robustness still cannot satisfy the demands in real world applications. In this work, we employed deep transfer learning to adapt the mismatch between source and target domains for robust hand gesture recognition. We systematically investigated 5 deep transfer learning algorithms on end-to-end convolutional neural network (CNN) architecture. Statistical machine learning methods with benchmark handcrafted features and general domain adaptation algorithms were also implemented as the strong baseline performance for the comparisons. The performance of the model under both inter-subject and inter-day applications was evaluated using high-density sEMG (HDsEMG) signals with 256 channels acquired from 36 subjects twice on 2 days in various rigorous validation protocols. Importantly, we examined why the deep transfer learning functioned by monitoring both the intra-domain and inter-domain mismatch between the learned features during model training. The results demonstrated that, without deep transfer learning, the intra-domain mismatch between samples under the same gestures decreased but the mismatch between domains (source and target domains) increased rapidly. The clustering of samples under the same gesture came at the expense of higher mismatch between different domains. With deep transfer learning applied, the inter-day and inter-subject performance of CNN was greatly improved (improved from 85.4% to 91.39%, and from 82.22% to 91.38%, respectively), which also outperformed benchmark statistical machine learning models with domain adaptation. In addition, the network structure of the proposed method is simple and can be easily implemented online. Overall, the findings in this work show great promise of transfer learning in hand gesture recognition under inter-subject and inter-day settings, significantly reducing the inconvenience of model re-calibration in practical use.

Cross-user Gesture Recognition Based on Deep Domain Adaptation Networks

RGC: Reliable Gesture Classification Via Wearables Using GANs-Based Data Augmentation.

Surface EMG-Based Inter-Session Gesture Recognition Enhanced by Deep Domain Adaptation.

Unsupervised Domain Adaptation by Causal Learning for Biometric Signal-based HCI

Semg-Based Gesture Recognition with Embedded Virtual Hand Poses and Adversarial Learning.

CrossGR

Cross-subject EMG hand gesture recognition based on dynamic domain generalization

Context-Aware Wireless-Based Cross-Domain Gesture Recognition

Domain Adaptation with Self-Guided Adaptive Sampling Strategy: Feature Alignment for Cross-User Myoelectric Pattern Recognition

Hand Gesture Recognition using Deep Feature Fusion Network based on Wearable Sensors

Deep end-to-end transfer learning for robust inter-subject and inter-day hand gesture recognition using surface EMG

Domain Adaptive Robotic Gesture Recognition with Unsupervised Kinematic-Visual Data Alignment

Adversary Helps: Gradient-based Device-Free Domain-Independent Gesture Recognition

XHAR: Deep Domain Adaptation for Human Activity Recognition with Smart Devices

A Robust and Real-Time Framework of Cross-Subject Myoelectric Control Model Calibration via Multi-Source Domain Adaptation

Optimization of inter-subject sEMG-based hand gesture recognition tasks using unsupervised domain adaptation techniques

Iterative Self-Training Based Domain Adaptation for Cross-User sEMG Gesture Recognition

Cross-user gesture recognition from sEMG signals using an optimal transport assisted student-teacher framework

Fusion of kinematic and physiological sensors for hand gesture recognition

An Event-Driven Spatiotemporal Domain Adaptation Method for DVS Gesture Recognition

Cross-Individual Gesture Recognition Based on Long Short-Term Memory Networks