Abstract:Sign Language Processing (SLP) is an interdisciplinary field comprised of Natural Language Processing (NLP) and Computer Vision. It is focused on the computational understanding, translation, and production of signed languages. Traditional approaches have often been constrained by the use of gloss-based systems that are both language-specific and inadequate for capturing the multidimensional nature of sign language. These limitations have hindered the development of technology capable of processing signed languages effectively. This thesis aims to revolutionize the field of SLP by proposing a simple paradigm that can bridge this existing technological gap. We propose the use of SignWiring, a universal sign language transcription notation system, to serve as an intermediary link between the visual-gestural modality of signed languages and text-based linguistic representations. We contribute foundational libraries and resources to the SLP community, thereby setting the stage for a more in-depth exploration of the tasks of sign language translation and production. These tasks encompass the translation of sign language from video to spoken language text and vice versa. Through empirical evaluations, we establish the efficacy of our transcription method as a pivot for enabling faster, more targeted research, that can lead to more natural and accurate translations across a range of languages. The universal nature of our transcription-based paradigm also paves the way for real-time, multilingual applications in SLP, thereby offering a more inclusive and accessible approach to language technology. This is a significant step toward universal accessibility, enabling a wider reach of AI-driven language technologies to include the deaf and hard-of-hearing community.

Synthesising Sign Language from semantics, approaching "from the target and back"

Example-Based Machine Translation from Text to a Hierarchical Representation of Sign Language

Everybody Sign Now: Translating Spoken Language to Photo Realistic Sign Language Video

A human-editable Sign Language representation for software editing---and a writing system?

DiffSign: AI-Assisted Generation of Customizable Sign Language Videos With Enhanced Realism

Empowering Sign Language Communication: Integrating Sentiment and Semantics for Facial Expression Synthesis

Toward an example-based machine translation from written text to ASL using virtual agent animation

Signs as Tokens: An Autoregressive Multilingual Sign Language Generator

Including Facial Expressions in Contextual Embeddings for Sign Language Generation

Neural Sign Actors: A diffusion model for 3D sign language production from text

Text2Sign: Towards Sign Language Production Using Neural Machine Translation and Generative Adversarial Networks

Reconstructing Signing Avatars From Video Using Linguistic Priors

Modeling Intensification for Sign Language Generation: A Computational Approach

Sign Stitching: A Novel Approach to Sign Language Production

Generating Signed Language Instructions in Large-Scale Dialogue Systems

Autoregressive Sign Language Production: A Gloss-Free Approach with Discrete Representations

Real-Time Multilingual Sign Language Processing

Modeling the Speed and Timing of American Sign Language to Generate Realistic Animations

Sign Language Production with Latent Motion Transformer

All You Need In Sign Language Production

The Sem-Lex Benchmark: Modeling ASL Signs and Their Phonemes