Abstract:Hand gesture recognition tasks based on surface electromyography (sEMG) are vital in human-computer interaction, speech detection, robot control, and rehabilitation applications. However, existing models, whether traditional machine learnings (ML) or other state-of-the-arts, are limited in the number of movements. Targeting a large number of gesture classes, more data features such as temporal information should be persisted as much as possible. In the field of sEMG-based recognitions, the recurrent convolutional neural network (RCNN) is an advanced method due to the sequential characteristic of sEMG signals. However, the invariance of the pooling layer damages important temporal information. In the all convolutional neural network (ACNN), because of the feature-mixing convolution operation, a same output can be received from completely different inputs. This paper proposes a concatenate feature fusion (CFF) strategy and a novel concatenate feature fusion recurrent convolutional neural network (CFF-RCNN). In CFF-RCNN, a max-pooling layer and a 2-stride convolutional layer are concatenated together to replace the conventional simple dimensionality reduction layer. The featurewise pooling operation serves as a signal amplitude detector without using any parameter. The feature-mixing convolution operation calculates the contextual information. Complete evaluations are made on both the accuracy and convergence speed of the CFF-RCNN. Experiments are conducted using three sEMG benchmark databases named DB1, DB2 and DB4 from the NinaPro database. With more than 50 gestures, the classification accuracies of the CFF-RCNN are 88.87% on DB1, 99.51% on DB2, and 99.29% on DB4. These accuracies are the highest compared with reported accuracies of machine learnings and other state-of-the-art methods. To achieve accuracies of 86%, 99% and 98% for the RCNN, the training time are 2353.686 s, 816.173 s and 731.771 s, respectively. However, for the CFF-RCNN to reach the same accuracies, it needs only 1727.415 s, 542.245 s and 576.734 s, corresponding to a reduction of 26.61%, 33.56% and 21.19% in training time. We concluded that the CFF-RCNN is an improved method when classifying a large number of hand gestures. The CFF strategy significantly improved model performance with higher accuracy and faster convergence as compared to traditional RCNN.

A Novel Resynchronization Procedure For Hand-Lips Fusion Applied To Continuous French Cued Speech Recognition

Re-Synchronization Using the Hand Preceding Model for Multi-Modal Fusion in Automatic Continuous Cued Speech Recognition

A New Re-synchronization Method Based Multi-modal Fusion for Automatic Continuous Cued Speech Recognition

A Novel Interpretable and Generalizable Re-synchronization Model for Cued Speech based on a Multi-Cuer Corpus

Speech neuromuscular decoding based on spectrogram images using conformal predictors with Bi-LSTM.

Multistream neural architectures for cued-speech recognition using a pre-trained visual feature extractor and constrained CTC decoding

Investigating the dynamics of hand and lips in French Cued Speech using attention mechanisms and CTC-based decoding

Computation and Parameter Efficient Multi-Modal Fusion Transformer for Cued Speech Recognition

Cross-Modal Mutual Learning for Cued Speech Recognition

Cross-Sentence Gloss Consistency for Continuous Sign Language Recognition

An Attention Self-supervised Contrastive Learning based Three-stage Model for Hand Shape Feature Representation in Cued Speech

Prior-aware Cross Modality Augmentation Learning for Continuous Sign Language Recognition

Monolingual Recognizers Fusion for Code-switching Speech Recognition

SlowFast Network for Continuous Sign Language Recognition

Spatio-Temporal Fusion Based Convolutional Sequence Learning for Lip Reading

Cross-Modal Knowledge Distillation Method for Automatic Cued Speech Recognition

Multimodal Fusion Based on LSTM and a Couple Conditional Hidden Markov Model for Chinese Sign Language Recognition

Automatic Detection of the Temporal Segmentation of Hand Movements in British English Cued Speech

Cuing Without Sharing: A Federated Cued Speech Recognition Framework via Mutual Knowledge Distillation

A novel concatenate feature fusion RCNN architecture for sEMG-based hand gesture recognition