Abstract:Finding ways to enable seamless communication between deaf and able-bodied individuals has been a challenging and pressing issue. This paper proposes a solution to this problem by designing a low-cost data glove that utilizes multiple inertial sensors with the purpose of achieving efficient and accurate sign language recognition. In this study, four machine learning models—decision tree (DT), support vector machine (SVM), K-nearest neighbor method (KNN), and random forest (RF)—were employed to recognize 20 different types of dynamic sign language data used by deaf individuals. Additionally, a proposed attention-based mechanism of long and short-term memory neural networks (Attention-BiLSTM) was utilized in the process. Furthermore, this study verifies the impact of the number and position of data glove nodes on the accuracy of recognizing complex dynamic sign language. Finally, the proposed method is compared with existing state-of-the-art algorithms using nine public datasets. The results indicate that both the Attention-BiLSTM and RF algorithms have the highest performance in recognizing the twenty dynamic sign language gestures, with an accuracy of 98.85% and 97.58%, respectively. This provides evidence for the feasibility of our proposed data glove and recognition methods. This study may serve as a valuable reference for the development of wearable sign language recognition devices and promote easier communication between deaf and able-bodied individuals.

Building Language Communication between Deaf People and Hearing Society through Multimodal Human-Computer Interface

Handtalker: A Multimodal Dialog System Using Sign Language and 3-D Virtual Human

Human-Computer Chinese Sign Language Interaction System

Improvement of Chinese sign language translation system based on collaboration of arm and finger sensing nodes

A Method to Synthesize Chinese Sign Language Based on Virtual Human Technologies

A Simple Sign Language Recognition System Based on Data Glove

A multimodal human-robot sign language interaction framework applied in social robots

SignFind: A Synchronized Sign Language and Chinese Character Teaching Game for Chinese Deaf Children Using Gesture Recognition

Multilingual Communication System with Deaf Individuals Utilizing Natural and Visual Languages

A Sign Language Recognition System Applied to Deaf-Mute Medical Consultation

Sign language translation system based on micro-inertial measurement units and ZigBee network

MDS: a multimodal-based dialog system

Design of Intelligent Human-Computer Interaction System for Hard of Hearing and Non-Disabled People

Dataglove for Sign Language Recognition of People with Hearing and Speech Impairment via Wearable Inertial Sensors

Sign Language Recognition and Translation: A Multidisciplined Approach From the Field of Artificial Intelligence

AI enabled sign language recognition and VR space bidirectional communication using triboelectric smart glove

Customizing Generated Signs and Voices of AI Avatars: Deaf-Centric Mixed-Reality Design for Deaf-Hearing Communication

HandTalker II: a Chinese sign language recognition and synthesis system

Real-Time Vision-Based Chinese Sign Language Recognition with Pose Estimation and Attention Network

American Sign Language Translation Using Wearable Inertial and Electromyography Sensors for Tracking Hand Movements and Facial Expressions

Text to Avatar in Multi-modal Human Computer Interface