Abstract:Communication between Deaf and hearing individuals remains a persistent challenge requiring attention to foster inclusivity. Despite notable efforts in the development of digital solutions for sign language recognition (SLR), several issues persist, such as cross-platform interoperability and strategies for tokenizing signs to enable continuous conversations and coherent sentence construction. To address such issues, this paper proposes a non-invasive Portuguese Sign Language (Língua Gestual Portuguesa or LGP) interpretation system-as-a-service, leveraging skeletal posture sequence inference powered by long-short term memory (LSTM) architectures. To address the scarcity of examples during machine learning (ML) model training, dataset augmentation strategies are explored. Additionally, a buffer-based interaction technique is introduced to facilitate LGP terms tokenization. This technique provides real-time feedback to users, allowing them to gauge the time remaining to complete a sign, which aids in the construction of grammatically coherent sentences based on inferred terms/words. To support human-like conditioning rules for interpretation, a large language model (LLM) service is integrated. Experiments reveal that LSTM-based neural networks, trained with 50 LGP terms and subjected to data augmentation, achieved accuracy levels ranging from 80% to 95.6%. Users unanimously reported a high level of intuition when using the buffer-based interaction strategy for terms/words tokenization. Furthermore, tests with an LLM-specifically ChatGPT-demonstrated promising semantic correlation rates in generated sentences, comparable to expected sentences.

Advancing Generative AI for Portuguese with Open Decoder Gervásio PT*

Advancing Neural Encoding of Portuguese with Transformer Albertina PT-*

Fostering the Ecosystem of Open Neural Encoders for Portuguese with Albertina PT* Family

GlórIA -- A Generative and Open Large Language Model for Portuguese

Tucano: Advancing Neural Text Generation for Portuguese

Open Sentence Embeddings for Portuguese with the Serafim PT* encoders family

Learning to Decode for Future Success

Empowering Deaf-Hearing Communication: Exploring Synergies between Predictive and Generative AI-Based Strategies towards (Portuguese) Sign Language Interpretation

Improving Portuguese Semantic Role Labeling with Transformers and Transfer Learning

From Brazilian Portuguese to European Portuguese

Lite Training Strategies for Portuguese-English and English-Portuguese Translation

Cabrita: closing the gap for foreign languages

A Legal Framework for Natural Language Processing Model Training in Portugal

Transformer Models for Brazilian Portuguese Question Generation: An Experimental Study

Open Generative Large Language Models for Galician

PORTULAN ExtraGLUE Datasets and Models: Kick-starting a Benchmark for the Neural Processing of Portuguese

PeLLE: Encoder-based language models for Brazilian Portuguese based on open data

Acoustic models of Brazilian Portuguese Speech based on Neural Transformers

Sabiá-2: A New Generation of Portuguese Large Language Models

Accelerating Transformer Inference for Translation via Parallel Decoding

Decoder Tuning: Efficient Language Understanding As Decoding