Abstract:Sign language is widely used to facilitate the communication process between deaf people and their surrounding environment. Sign language, like most other languages, is considered a complex language which cannot be mastered easily. Thus, technology can be used as an assistive tool to solve the difficulties and challenges that deaf people face during interactions with society. In this study, an automatic bidirectional translation framework for Arabic Sign Language (ArSL) is designed to assist both deaf and ordinary people to communicate and express themselves easily. Two main modules were intended to translate Arabic sign images into text by utilizing different transfer learning models and to translate the input text into Arabic sign images. A prototype was implemented based on the proposed framework by using several pre-trained convolutional neural network (CNN)-based deep learning models, including the DenseNet121, ResNet152, MobileNetV2, Xception, InceptionV3, NASNetLarge, VGG19, and VGG16 models. A fuzzy string matching score method, as a novel concept, was employed to translate the input text from ordinary people into appropriate sign language images. The dataset was constructed with specific criteria to obtain 7030 images for 14 classes captured from both deaf and ordinary people locally. The prototype was developed to conduct the experiments on the collected ArSL dataset using the utilized CNN deep learning models. The experimental results were evaluated using standard measurement metrics such as accuracy, precision, recall, and F1-score. The performance and efficiency of the ArSL prototype were assessed using a test set of an 80:20 splitting procedure, obtaining accuracy results from the highest to the lowest rates with average classification time in seconds for each utilized model, including (VGG16, 98.65%, 72.5), (MobileNetV2, 98.51%, 100.19), (VGG19, 98.22%, 77.16), (DenseNet121, 98.15%, 80.44), (Xception, 96.44%, 72.54), (NASNetLarge, 96.23%, 84.96), (InceptionV3, 94.31%, 76.98), and (ResNet152, 47.23%, 98.51). The fuzzy matching score is mathematically validated by computing the distance between the input and associative dictionary words. The study results showed the prototype's ability to successfully translate Arabic sign images into Arabic text and vice versa, with the highest accuracy. This study proves the ability to develop a robust and efficient real-time bidirectional ArSL translation system using deep learning models and the fuzzy string matching score method.

Arabic sign language intelligent translator

An intelligent system to help deaf students learn Arabic Sign Language

A Bidirectional Arabic Sign Language Framework Using Deep Learning and Fuzzy Matching Score

Design of Arabic Sign Language Recognition Model

Advanced Arabic Alphabet Sign Language Recognition Using Transfer Learning and Transformer Models

Gesture based Arabic Sign Language Recognition for Impaired People based on Convolution Neural Network

A Real Time Arabic Sign Language Alphabets (ArSLA) Recognition Model Using Deep Learning Architecture

Real-Time Arabic Sign Language Recognition Using a Hybrid Deep Learning Model

A real-time Arabic avatar for deaf–mute community using attention mechanism

Machine Learning and Deep Learning Approaches for Arabic Sign Language Recognition: A Decade Systematic Literature Review

Signer-Independent Arabic Sign Language Recognition System Using Deep Learning Model

SignsWorld; Deeping Into the Silence World and Hearing Its Signs (State of the Art)

Isolated Arabic Sign Language Recognition Using A Transformer-based Model and Landmark Keypoints

Efhamni: A Deep Learning-Based Saudi Sign Language Recognition Application

3D gesture segmentation for word-level Arabic sign language using large-scale RGB video sequences and autoencoder convolutional networks

Enhancing Arabic Sign Language Interpretation: Leveraging Convolutional Neural Networks and Transfer Learning

Real-time mobile application for Arabic sign alphabet recognition using pre-trained CNN

A multi-lingual sign language recognition system using machine learning

Towards A Sign Language Gloss Representation Of Modern Standard Arabic

Automated sign language detection and classification using reptile search algorithm with hybrid deep learning

Human-Computer Interaction with Hand Gesture Recognition Using ResNet and MobileNet