Abstract:Sign languages are gesture-based languages used by the deaf community of the world. Every country has a different sign language and there are more than 200 sign languages in the world. American Sign Language (ASL), British Sign Language (BSL), and German Sign Language (DGS) are well-studied sign languages. Due to their different grammatical structure and word order the deaf people feel difficulty in reading and understanding the written text in natural languages. In order to enhance the cognitive ability of the deaf subjects some translation models have been developed for these languages that translate natural language text into corresponding sign language gestures. Most of the earlier natural to sign language translation models rely on rule-based approaches. Recently, some neural machine translation models have been proposed for ASL, BSL, DGS, and Arabic sign language. However, most of these models have low accuracy scores. This research provides an improved and novel multi-stack RNN-based neural machine translation model for natural to sign language translation. The proposed model is based on encoder–decoder architecture and incorporates attention mechanism and embeddings to improve the quality of results. Rigorous experimentation has been performed to compare the proposed multi-stack RNN-based model with baseline models. The experiments have been conducted using a sizeable translation corpus comprising of nearly 50,000 sentences for Pakistan Sign Language (PSL). The performance of the proposed neural machine translation model for PSL has been evaluated with the help of well-established evaluation measures including Bilingual Evaluation Understudy Score (BLEU), and Word Error Rate (WER). The results show that multi-stacked gated recurrent unit-based RNN model that employs Bahdanau attention mechanism and GloVe embedding performed the best showing the BLEU score of 0.83 and WER 0.17, which outperform the existing translation models. The proposed model has been exposed through a software system that converts the translated sentences into PSL gestures using an avatar. The evaluation of the usability has also been performed to see how effectively the avatar-based output helps compensating the cognitive hearing deficit for the deaf people. The results show that it works well for different granularity levels.

RNN-LSTM-GRU based language transformation

Urdu-English Machine Transliteration using Neural Networks

Sequence to Sequence Networks for Roman-Urdu to Urdu Transliteration

Machine Translation System Using Deep Learning for English to Urdu

A diachronic study determining syntactic and semantic features of Urdu-English neural machine translation

Long short-term memory recurrent neural network architectures for Urdu acoustic modeling

A deep learning approach for Named Entity Recognition in Urdu language

Exploiting Transliterated Words for Finding Similarity in Inter-Language News Articles using Machine Learning

Context based Roman-Urdu to Urdu Script Transliteration System

Recognition of offline handwritten Urdu characters using RNN and LSTM models

Machine learning based framework for fine-grained word segmentation and enhanced text normalization for low resourced language

A Permuted Autoregressive Approach to Word-Level Recognition for Urdu Digital Text

UTSA: Urdu Text Sentiment Analysis Using Deep Learning Methods

Generalists vs. Specialists: Evaluating Large Language Models for Urdu

Neural Machine Translation for Low-Resource Languages: A Survey

Towards achieving a delicate blending between rule-based translator and neural machine translator

Neural Machine Translation model for University Email Application

From Statistical Methods to Pre-Trained Models; A Survey on Automatic Speech Recognition for Resource Scarce Urdu Language

Deep Sentiments Analysis for Roman Urdu Dataset Using Faster Recurrent Convolutional Neural Network Model

Attention-Based RU-BiLSTM Sentiment Analysis Model for Roman Urdu

A multi-stack RNN-based neural machine translation model for English to Pakistan sign language translation