Abstract:Gesture recognition technology based on surface electromyography (sEMG) has attracted widespread attention and research. However, most studies have been limited to gesture recognition at a single force level, ignoring the importance of force level recognition during interaction, leading to limitations in its application. Therefore, an attention-guided multiscale multitask learning network (AGMMLN) is proposed in this article. With the Log-Mel time-frequency spectrum of multichannel nonstationary sEMG signals as input, AGMMLN can recognize gestures and force levels simultaneously and accurately. Given the limitation of fixed-scale analysis of classical convolutional neural networks (CNNs), a multiscale convolution module (MSCM) is proposed to simultaneously extract both fine-grained local features and extensive global features in the temporal spectrum. To filter out redundant information in multiscale features and enhance key features, a group convolution module (GCM) is proposed to facilitate nonlinear interactions between feature channels and enhance feature representation. In addition, to automatically select features beneficial to a specific task from the rich set of shared features, a feature attention module (FAM) is proposed to enhance useful features and suppress useless ones. Experiments are conducted on a self-collected able-bodied dataset and a publicly available amputee dataset to validate the effectiveness of the proposed network. The experimental results show that the proposed method achieves 95.19% and 89.91%, and 90.78% and 75.96% accuracy in the simultaneous recognition of gesture and force levels in able-bodied and amputee subjects, respectively, which is significantly superior to the other compared methods.

Multimodal Gesture Recognition Using Multi-stream Recurrent Neural Network

Multimodal Gesture Recognition Using 3-D Convolution and Convolutional LSTM

Multi-modal learning for gesture recognition

Multimode Gesture Recognition Algorithm Based on Convolutional Long Short-Term Memory Network

Deep Dynamic Neural Networks for Multimodal Gesture Segmentation and Recognition

Multi-stream Fusion Network for Continuous Gesture Recognition Based on Semg

Multimodal Spatiotemporal Feature Map for Dynamic Gesture Recognition

Surface Electromyography-based Gesture Recognition by Multi-view Deep Learning.

Multimodal Gesture Recognition Based on the ResC3D Network.

Multi-modal fusion for robust hand gesture recognition based on heterogeneous networks

Multi-Task and Multi-Modal Learning for RGB Dynamic Gesture Recognition

Multi-Scale Attention 3D Convolutional Network for Multimodal Gesture Recognition

Large-scale Multimodal Gesture Segmentation and Recognition Based on Convolutional Neural Networks

Multimodal Gesture Recognition Based on Attention Slow-Fast Fusion Networks

Searching Multi-Rate and Multi-Modal Temporal Enhanced Networks for Gesture Recognition

Study on Gesture Recognition Method with Two-Stream Residual Network Fusing sEMG Signals and Acceleration Signals

Multi-Sensor Guided Hand Gesture Recognition for a Teleoperated Robot Using a Recurrent Neural Network

Large-Scale Multimodal Gesture Recognition Using Heterogeneous Networks

AGMMLN: an Attention-Guided Multiscale Multitask Learning Network for Simultaneous Gesture and Force Level Recognition

Multistage Spatial Attention-Based Neural Network for Hand Gesture Recognition

A Multimodal Dynamic Hand Gesture Recognition Based on Radar–Vision Fusion