Abstract:Sign Language Recognition (SLR) represents a revolutionary technology aiming to establish communication between hearing impaired and non-hearing impaired communities, surpassing traditional interpreter-based approaches. Existing efforts in automatic sign recognition predominantly rely on hand skeleton joint information, steering clear of image pixels to address challenges like partial occlusion and redundant backgrounds. Many researchers have been working to develop automatic sign recognition using hand skeleton joint information instead of image pixels to overcome partial occlusion and redundant background problems. However, body motion and facial expression play an essential role in increasing the inner gesture variance in expressing sign language emotion besides hand information for large-scale sign word datasets. Recently, some researchers have been working to develop muti-gesture-based SLR recognition systems, but their performance accuracy and efficiency are unsatisfactory for real-time deployment. Addressing these limitations, we propose a novel approach, a two-stream multistage graph convolution with attention and residual connection (GCAR) designed to extract spatial-temporal contextual information. The multistage GCAR system, incorporating a channel attention module, dynamically enhances attention levels, particularly for non-connected skeleton points during specific events within spatial-temporal features. The methodology involves capturing joint skeleton points and motion, offering a comprehensive understanding of a person's entire body movement during sign language gestures and feeding this information into two streams. In the first stream, joint key features undergo processing through sep-TCN, graph convolution, deep learning layer, and a channel attention module across multiple stages, generating intricate spatial-temporal features in sign language gestures. Simultaneously, the joint motion is processed in the second stream, mirroring the steps of the first branch. The fusion of these two features yields the final feature vector, which is then fed into the classification module. The model excels in capturing discriminative structural displacements and short-range dependencies by leveraging unified joint features projected onto a high-dimensional space. Owing to the effectiveness of these features, the proposed method achieved significant accuracies: 90.31%, 94.10%, 99.75%, and 34.41%, for the WLASL, PSL, MSL, and ASLLVD large-scale datasets, respectively, with 0.69 million parameters. The high-performance accuracy, coupled with stable computational complexity, demonstrates the superiority of the proposed model. This innovative approach is anticipated to redefine the landscape of sign language recognition, setting a new standard in the field.

MediSign: An Attention-Based CNN-BiLSTM Approach of Classifying Word Level Signs for Patient-Doctor Interaction in Hearing Impaired Community

Intelligent Malaysian Sign Language Translation System Using Convolutional-Based Attention Module with Residual Network

Bengali-Sign: A Machine Learning-Based Bengali Sign Language Interpretation for Deaf and Non-Verbal People

Intelligent real-time Arabic sign language classification using attention-based inception and BiLSTM

Signer-Independent Arabic Sign Language Recognition System Using Deep Learning Model

Enhancing Arabic Sign Language Interpretation: Leveraging Convolutional Neural Networks and Transfer Learning

Bangla Sign Language (BdSL) Alphabets and Numerals Classification Using a Deep Learning Model

Sign Language Conversation Interpretation Using Wearable Sensors and Machine Learning

Sign Language Recognition Using Graph and General Deep Neural Network Based on Large Scale Dataset

A Sign Language Recognition System Applied to Deaf-Mute Medical Consultation

BAUST Lipi: A BdSL Dataset with Deep Learning Based Bangla Sign Language Recognition

Non-Touch Sign Word Recognition Based on Dynamic Hand Gesture Using Hybrid Segmentation and CNN Feature Fusion

A multi-lingual sign language recognition system using machine learning

Connecting the Dots: Leveraging Spatio-Temporal Graph Neural Networks for Accurate Bangla Sign Language Recognition

Real-Time Arabic Sign Language Recognition Using a Hybrid Deep Learning Model

A wearable system for sign language recognition enabled by a convolutional neural network

Automated Sign to Speech Conversion Model using Deep Learning

1DCNNTrans: BISINDO Sign Language Interpreters in Improving the Inclusiveness of Public Services

Gesture based Arabic Sign Language Recognition for Impaired People based on Convolution Neural Network

AI-Based Sensory Glove System to Recognize Bengali Sign Language (BaSL)

Spatial–temporal attention with graph and general neural network-based sign language recognition