Abstract:Abstract Speech-impaired people use Sign Language (SL), an efficient natural form of communication, all over the world. This paper aims to use deep learning technology in the realm of SL translation and identification. In order to ease communication between hearing-impaired and sighted individuals and to enable the social inclusion of hearing-impaired people in their daily lives, it presents a transformer as a neural machine translation model. The article details the creation of a machine translation system that converts Arabic audio and text into Arabic Sign Language (ArSL) automatically. It does this by utilizing an animated character to produce the correct sign for each spoken word. Since Arabic has few resources, it was challenging to obtain an Arabic-Sign dataset, so we created our own Arabic–Arabic sign gloss, which consists of 12,187 pairs, to train the model. We use bidirectional encoder representations from transformers as an embedding layer to interpret input text tokens and represent an appropriate natural language vector space for deep learning models. To represent the structure of each Arabic word, the Ferasa Part-of-Speech Tagging module was used and then the extracted rules from the ArSL structure were applied. This paper shows a detailed description of a natural language translator (for converting an Arabic word sequence into a sequence of signs belonging to the ArSL) and a 2D avatar animation module (for playing back the signs). In our prototype, we train the software-based module using the attention mechanism. The evaluation was carried out in our developed Arabic sentences with the corresponding Arabic gloss. The proposed model achieves promising results and indicates significant improvements to direct communication between hearing and deaf people, with a training accuracy of 94.71% and an 87.04% testing accuracy for Arabic–Arabic sign gloss translation.

Statistical Sign Language Machine Translation: from English written text to American Sign Language Gloss

Toward an example-based machine translation from written text to ASL using virtual agent animation

A real-time Arabic avatar for deaf–mute community using attention mechanism

American Sign Language Video to Text Translation

Arabic sign language intelligent translator

English text to Indian Sign Language Machine Translation: A Rule Based Method

American Sign Language Translation Using Wearable Inertial and Electromyography Sensors for Tracking Hand Movements and Facial Expressions

Machine Translation between Spoken Languages and Signed Languages Represented in SignWriting

Bidirectional American Sign Language to English Translation

Translation System for Sign Language Learning

SmartASL

An Open-Source Gloss-Based Baseline for Spoken to Signed Language Translation

Text2Sign: Towards Sign Language Production Using Neural Machine Translation and Generative Adversarial Networks

Enhancing Bidirectional Sign Language Communication: Integrating YOLOv8 and NLP for Real-Time Gesture Recognition & Translation

An intelligent system to help deaf students learn Arabic Sign Language

Machine translation from text to sign language: a systematic review

SignSpeak: Open-Source Time Series Classification for ASL Translation

Data Augmentation for Sign Language Gloss Translation

Sign Language Recognition and Translation: A Multidisciplined Approach From the Field of Artificial Intelligence

Example-Based Machine Translation from Text to a Hierarchical Representation of Sign Language

Bengali to Assamese Statistical Machine Translation using Moses (Corpus Based)