A new architecture for hand-worn Sign language to Speech translator

Sai Charan Bodda,Palki Gupta,Gaurav Joshi,Ayush Chaturvedi

DOI: https://doi.org/10.48550/arXiv.2009.03988

IF: 6.4588

2020-09-08

Human-Computer Interaction

Abstract:People with speech and hearing impairments often rely on sign language to communicate with others but most of the general population cannot understand sign language and sign language itself is a difficult language to learn, so there is a definite need for technologies to translate sign language to speech. In this paper, we describe the design and implementation of Smart glove, a hand-worn hardware device capable of translating American Sign Language gestures into English speech by tracking the finger's orientation, gestures and hand motion. It uses hardware sensors like Flex, Accelerometer and gyroscope and intelligent software to capture and translate the gestures into speech. This paper explains the translation of both Alphabet and Word gestures. New approaches and algorithms are proposed and implemented to address hardware-dependent issues in existing glove based designs. The whole device is designed to be modular with distributed processing units to encourage modular enhancement, reducing complexity, and interrelation between subsystems.Decision Trees are used in gesture recognition and error correction. We hope that the henceforth mentioned design and architecture would be the basis for the advancement in research related to sensor-based sign language translation along with research for smart glove and cybernetic accessories.