Abstract:Sign languages are gesture-based languages used by the deaf community of the world. Every country has a different sign language and there are more than 200 sign languages in the world. American Sign Language (ASL), British Sign Language (BSL), and German Sign Language (DGS) are well-studied sign languages. Due to their different grammatical structure and word order the deaf people feel difficulty in reading and understanding the written text in natural languages. In order to enhance the cognitive ability of the deaf subjects some translation models have been developed for these languages that translate natural language text into corresponding sign language gestures. Most of the earlier natural to sign language translation models rely on rule-based approaches. Recently, some neural machine translation models have been proposed for ASL, BSL, DGS, and Arabic sign language. However, most of these models have low accuracy scores. This research provides an improved and novel multi-stack RNN-based neural machine translation model for natural to sign language translation. The proposed model is based on encoder–decoder architecture and incorporates attention mechanism and embeddings to improve the quality of results. Rigorous experimentation has been performed to compare the proposed multi-stack RNN-based model with baseline models. The experiments have been conducted using a sizeable translation corpus comprising of nearly 50,000 sentences for Pakistan Sign Language (PSL). The performance of the proposed neural machine translation model for PSL has been evaluated with the help of well-established evaluation measures including Bilingual Evaluation Understudy Score (BLEU), and Word Error Rate (WER). The results show that multi-stacked gated recurrent unit-based RNN model that employs Bahdanau attention mechanism and GloVe embedding performed the best showing the BLEU score of 0.83 and WER 0.17, which outperform the existing translation models. The proposed model has been exposed through a software system that converts the translated sentences into PSL gestures using an avatar. The evaluation of the usability has also been performed to see how effectively the avatar-based output helps compensating the cognitive hearing deficit for the deaf people. The results show that it works well for different granularity levels.

Phrase Based and Neural Network Translation for Text Transliteration from Arabic to Indonesia

Urdu-English Machine Transliteration using Neural Networks

First Result on Arabic Neural Machine Translation

Exploiting Transliterated Words for Finding Similarity in Inter-Language News Articles using Machine Learning

A Comparative Study of Deep Learning Approaches for Arabic Language Processing

A multi-stack RNN-based neural machine translation model for English to Pakistan sign language translation

Arabic Language Opinion Mining Based on Long Short-Term Memory (LSTM)

RNN-LSTM-GRU based language transformation

Speech to Text Translation for Malay Language

Transliteration of Judeo-Arabic Texts into Arabic Script Using Recurrent Neural Networks

Language Detection for Transliterated Content

Abstractive Arabic Text Summarization Based on Deep Learning

Extending a model for ontology-based Arabic-English machine translation

Content-Localization based Neural Machine Translation for Informal Dialectal Arabic: Spanish/French to Levantine/Gulf Arabic

Phrase translation using a bilingual dictionary and n-gram data: A case study from Vietnamese to English

Improving neural machine translation for low resource languages through non-parallel corpora: a case study of Egyptian dialect to modern standard Arabic translation

Recognition of Arabic Air-Written Letters: Machine Learning, Convolutional Neural Networks, and Optical Character Recognition (OCR) Techniques

Neural Network Transduction Models in Transliteration Generation

Error Analysis of Pretrained Language Models (PLMs) in English-to-Arabic Machine Translation

A real-time Arabic avatar for deaf–mute community using attention mechanism

Sequence to Sequence Networks for Roman-Urdu to Urdu Transliteration