Abstract:Implementation of automatic sign language translation software in the process of social inclusion of people with hearing impairment is an important task. Social inclusion for people with hearing disabilities is an acute problem that must be solved in the context of the development of IT technologies and legislative initiatives that ensure the rights of people with disabilities and their equal opportunities. This substantiates the relevance of the research of assistive technologies, in the context of software tools, such as the process of social inclusion of people with severe hearing impairment in society. The subject of research is methods of automated sign language translation using intelligent technologies. The purpose of the work is the development and research of sign language automation methods to improve the quality of life of people with hearing impairments in accordance with the "Goals of Sustainable Development of Ukraine" (in the "Reduction of Inequality" part). The main tasks of the research are the development and testing of methods of converting sign language into text, converting text into sign language, as well as automating translation from one sign language to another sign language using modern intelligent technologies. Neural network modeling and 3D animation methods were used to solve these problems. The following results were obtained in the work: the main problems and tasks of social inclusion for people with hearing impairments were identified; a comparative analysis of modern methods and software platforms of automatic sign language translation was carried out; a system combining the SL-to-Text method is proposed and investigated; the Text-to-SL method using 3D animation to generate sign language concepts; the method of generating a 3D-animated gesture from video recordings; method of implementing the Sign Language1 to Sign Language2 technology. For gesture recognition, a convolutional neural network model is used, which is trained using imported and system-generated datasets of video gestures. The trained model has a high recognition accuracy (98.52%). The creation of a 3D model for displaying the gesture on the screen and its processing took place in the Unity 3D environment. The structure of the project, executive and auxiliary files used to build 3D animation for the generation of sign language concepts includes: event handler files; display results according to which they carry information about the position of the tracked points of the body; files that store the characteristics of materials that have been added to certain body mapping points. Conclusions: the proposed methods of automated translation have practical significance, which is confirmed by the demo versions of the software applications "Sign Language to Text" and "Text to Sign Language". A promising direction for continuing research on the topic of the work is the improvement of SL1-to-SL2 methods, the creation of open datasets of video gestures, the joining of scientists and developers to fill dictionaries with concepts of various sign languages.

Regression Analysis of Demographic and Technology-Experience Factors Influencing Acceptance of Sign Language Animation

Empirical Investigation of Users' Preferred Timing Parameters for American Sign Language Animations

Do predictability factors towards signing avatars hold across cultures?

Getting a Sign in Edgewise: User-Centered Design Considerations in Creating a Signed Language Mentoring Management System

Automatic sign language translation system using neural network technologies and 3D animation

Toward an example-based machine translation from written text to ASL using virtual agent animation

Translation System for Sign Language Learning

Automated 3D sign language caption generation for video

Modeling the Speed and Timing of American Sign Language to Generate Realistic Animations

Assessment of Sign Language-Based versus Touch-Based Input for Deaf Users Interacting with Intelligent Personal Assistants

DiffSign: AI-Assisted Generation of Customizable Sign Language Videos With Enhanced Realism

An exploratory user study towards developing a unified, comprehensive assessment apparatus for deaf signers, specifically tailored for signing avatars evaluation: challenges, findings, and recommendations

Assessing the Need for a Wearable Sign Language Recognition Device for Deaf Individuals: Results from a National Questionnaire

Chinese Sign Language Animation System on Mobile Devices

Sign Language Recognition and Translation: A Multidisciplined Approach From the Field of Artificial Intelligence

Landscape of sign language research based on smartphone apps: coherent literature analysis, motivations, open challenges, recommendations and future directions for app assessment

Attitudes Toward Signing Avatars Vary Depending on Hearing Status, Age of Signed Language Acquisition, and Avatar Type

Towards Inclusive Video Commenting: Introducing Signmaku for the Deaf and Hard-of-Hearing

The Interactive Video-Questionnaire: a new technology for interviewing deaf persons

Design and Implementation of a Virtual 3D Educational Environment to improve Deaf Education

A real-time Artificial Intelligence system for learning Sign Language