Abstract:Sign language recognition (SLR) is an important communication tool between the deaf and the external world. It is highly necessary to develop a worldwide continuous and large-vocabulary-scale SLR system for practical usage. In this paper, we propose a novel phonology- and radical-coded Chinese SLR framework to demonstrate the feasibility of continuous SLR using accelerometer (ACC) and surface electromyography (sEMG) sensors. The continuous Chinese characters, consisting of coded sign gestures, are first segmented into active segments using EMG signals by means of moving average algorithm. Then, features of each component are extracted from both ACC and sEMG signals of active segments (i.e., palm orientation represented by the mean and variance of ACC signals, hand movement represented by the fixed-point ACC sequence, and hand shape represented by both the mean absolute value (MAV) and autoregressive model coefficients (ARs)). Afterwards, palm orientation is first classified, distinguishing “Palm Downward” sign gestures from “Palm Inward” ones. Only the “Palm Inward” gestures are sent for further hand movement and hand shape recognition by dynamic time warping (DTW) algorithm and hidden Markov models (HMM) respectively. Finally, component recognition results are integrated to identify one certain coded gesture. Experimental results demonstrate that the proposed SLR framework with a vocabulary scale of 223 characters can achieve an averaged recognition accuracy of 96.01% ± 0.83% for coded gesture recognition tasks and 92.73% ± 1.47% for character recognition tasks. Besides, it demonstrats that sEMG signals are rather consistent for a given hand shape independent of hand movements. Hence, the number of training samples will not be significantly increased when the vocabulary scale increases, since not only the number of the completely new proposed coded gestures is constant and limited, but also the transition movement which connects successive signs needs no training samples to model even though the same coded gesture performed in different characters. This work opens up a possible new way to realize a practical Chinese SLR system.

A Sign Language Recognition Method Based on Multi-sensor Information

A real-time hand gesture recognition method

Gesture Recognition with a 3-D Accelerometer

Design and Implementation of Gesture Recognition System Based on Flex Sensors

Gesture recognition using combination of acceleration sensor and images for casual communication between robots and humans

Multimodal Gesture Recognition Based On Choquet Integral

A Novel Phonology- and Radical-Coded Chinese Sign Language Recognition Framework Using Accelerometer and Surface Electromyography Sensors

Multimodal Hand/finger Movement Sensing and Fuzzy Encoding for Data‐efficient Universal Sign Language Recognition

Sign language translation system based on micro-inertial measurement units and ZigBee network

Improvement of Chinese Sign Language Translation System Based on Multi-Node Micro Inertial Measurement Unit

Hand gesture recognition using smooth wavelet packet transformation and hybrid CNN based on surface EMG and accelerometer signal

Dataglove for Sign Language Recognition of People with Hearing and Speech Impairment via Wearable Inertial Sensors

A Static Sign Language Recognition Method Enhanced with Self-Attention Mechanisms

Multiuser gesture recognition using sEMG signals via canonical correlation analysis and optimal transport

Study on Gesture Recognition Method with Two-Stream Residual Network Fusing sEMG Signals and Acceleration Signals

Hand Gesture Recognition using Deep Feature Fusion Network based on Wearable Sensors

Fusion of kinematic and physiological sensors for hand gesture recognition

An adaptive learning method for long-term gesture recognition based on surface electromyography

A Fast Sign Word Recognition Method for Chinese Sign Language

A wearable sensor-based dynamic gesture recognition model via broad attention learning

Improvement of Chinese sign language translation system based on collaboration of arm and finger sensing nodes