sEMG-based Hand Gesture Recognition with Deep Learning

Marcello Zanghieri

2023-06-19

Abstract:Hand gesture recognition based on surface electromyographic (sEMG) signals is a promising approach for developing Human-Machine Interfaces (HMIs) with a natural control, such as intuitive robot interfaces or poly-articulated prostheses. However, real-world applications are limited by reliability problems due to motion artefacts, postural and temporal variability, and sensor re-positioning. This master thesis is the first application of deep learning on the Unibo-INAIL dataset, the first public sEMG dataset exploring the variability between subjects, sessions and arm postures by collecting data over 8 sessions of each of 7 able-bodied subjects executing 6 hand gestures in 4 arm postures. Recent studies address variability with strategies based on training set composition, which improve inter-posture and inter-day generalization of non-deep machine learning classifiers, among which the RBF-kernel SVM yields the highest accuracy. The deep architecture realized in this work is a 1d-CNN inspired by a 2d-CNN reported to perform well on other public benchmark databases. On this 1d-CNN, various training strategies based on training set composition were implemented and tested. Multi-session training proves to yield higher inter-session validation accuracies than single-session training. Two-posture training proves the best postural training (proving the benefit of training on more than one posture) and yields 81.2% inter-posture test accuracy. Five-day training proves the best multi-day training, yielding 75.9% inter-day test accuracy. All results are close to the baseline. Moreover, the results of multi-day training highlight the phenomenon of user adaptation, indicating that training should also prioritize recent data. Though not better than the baseline, the achieved classification accuracies rightfully place the 1d-CNN among the candidates for further research.

Signal Processing,Machine Learning

What problem does this paper attempt to address?

The problem this paper attempts to address is the reliability of gesture recognition based on surface electromyography (sEMG) in practical applications. Specifically, the paper focuses on the following aspects:

1. **Motion Artifacts**: Signal distortion caused by limb movement.
2. **Posture and Time Variability**: Differences in data collected from different postures and at different times.
3. **Sensor Relocation**: Changes in sensor position each time it is used.

These issues limit the application of sEMG-based gesture recognition technology in real-world scenarios, especially in situations requiring high reliability and stability, such as intuitive robotic interfaces or multi-joint prosthesis control.

To address these challenges, the paper applies deep learning to the Unibo-INAIL dataset for the first time. This is an open sEMG dataset that explores variability between subjects, sessions, and arm postures. By implementing and testing different training strategies, the paper aims to improve the model's generalization ability across different postures and days, thereby enhancing the reliability and accuracy of gesture recognition.