Abstract:Interfaces based on surface electromyography (sEMG) and inertial measurement units (IMUs) enable users to interact with computers in a natural and intuitive way through hand gestures. sEMG or IMU sensor-based approaches can provide muscle electrical activity or motion information to recognize gestures. However, the existing methods recognize static and dynamic gestures separately and hierarchically. In addition, the variability in sEMG data among different subjects limits the performance of sensor-based human–computer interfaces. To address these limitations, this article proposes a multistream convolutional neural network (CNN) architecture with a fine-tuning transfer strategy. The multistream architecture explores the complementary nature of sEMG and IMU signals and achieves real-time recognition of dynamic and static gestures in a nonhierarchical way. In addition, a new dataset including seven static gestures and six dynamic gestures is designed for evaluation. The proposed method is verified by experiments on six subjects and is compared with previous methods on the same dataset. Experimental results reveal that the transfer strategy significantly improves recognition accuracy from 89.99% to 98.49%. Meanwhile, by learning the fusion features of sEMG and IMU signals, the recognition accuracy is enhanced from 49.54% (sEMG) and 59.91% (IMU) to 89.99%. In addition, the accuracy and latency of this method for real-time recognition are 98.12% and 103 ms, respectively. These results demonstrate that the proposed multistream CNN model accurately recognizes these static and dynamic gestures online in a nonhierarchical way and can effectively utilize the complementarity between sEMG and IMU signals.

A Semg-Based Hand Gesture Recognition Using Mulit-channel CNN and MLP.

Surface Electromyography-based Gesture Recognition by Multi-view Deep Learning.

Hand Gesture Recognition Based Improved Multi-Channels CNN Architecture Using EMG Sensors

A multi-stream convolutional neural network for sEMG-based gesture recognition in muscle-computer interface

Gesture Classification in Electromyography Signals for Real-Time Prosthetic Hand Control Using a Convolutional Neural Network-Enhanced Channel Attention Model

Multi-gesture Recognition Method of Single-Channel EMG Signal Based on Deep Learning

Classification of multichannel surface-electromyography signals based on convolutional neural networks

Machine Learning-based sEMG Signal Classification for Hand Gesture Recognition

Channel-distribution Hybrid Deep Learning for Semg-Based Gesture Recognition.

A Global and Local Feature fused CNN architecture for the sEMG-based hand gesture recognition

Semg-Based Gesture Recognition Using Deep Learning from Noisy Labels.

A novel attention-based hybrid CNN-RNN architecture for sEMG-based gesture recognition

Deep Neural Network for Electromyography Signal Classification Via Wearable Sensors.

Hand Gesture Recognition Using Sparse Autoencoder-based Deep Neural Network Based on Electromyography Measurements

Gesture recognition based on surface electromyography‐feature image

sEMG and IMU Data-Based Hand Gesture Recognition Method Using Multistream CNN With a Fine-Tuning Transfer Framework

Semg-Based Gesture Recognition with Embedded Virtual Hand Poses and Adversarial Learning.

Gesture recognition based on surface electromyography <scp>‐feature</scp> image

Classification of Hand Gestures Based on Single-channel Semg Decomposition

A Hybrid CNN-LSTM Network for Hand Gesture Recognition with Surface EMG Signals

Hand Gesture Recognition Based on Deep Learning Method.