Abstract:Fingerspelling in sign language has been the means of communicating technical terms and proper nouns when they do not have dedicated sign language gestures. Automatic recognition of fingerspelling can help resolve communication barriers when interacting with deaf people. The main challenges prevalent in fingerspelling recognition are the ambiguity in the gestures and strong articulation of the hands. The automatic recognition model should address high inter-class visual similarity and high intra-class variation in the gestures. Most of the existing research in fingerspelling recognition has focused on the dataset collected in a controlled environment. The recent collection of a large-scale annotated fingerspelling dataset in the wild, from social media and online platforms, captures the challenges in a real-world scenario. In this work, we propose a fine-grained visual attention mechanism using the Transformer model for the sequence-to-sequence prediction task in the wild dataset. The fine-grained attention is achieved by utilizing the change in motion of the video frames (optical flow) in sequential context-based attention along with a Transformer encoder model. The unsegmented continuous video dataset is jointly trained by balancing the Connectionist Temporal Classification (CTC) loss and the maximum-entropy loss. The proposed approach can capture better fine-grained attention in a single iteration. Experiment evaluations show that it outperforms the state-of-the-art approaches.

CHINESE FINGER ALPHABET FLOW RECOGNITION SYSTEM BASED ON MULTI-FEATURES AND MULTI-CLASSIFIER

The Recognition of Finger-Spelling for Chinese Sign Language.

The recognition of finger-spelling alphabets for Chinese sign language

SmartHandwriting: Handwritten Chinese Character Recognition with Smartwatch.

SmartWriting: Pen-Holding Gesture Recognition with Smartwatch

Recognition of A Limited Chinese Character Set Based on Improved CLAFIC-LSM Algorithm

A Continuous Chinese Sign Language Recognition System

An eight-layer convolutional neural network with stochastic pooling, batch normalization and dropout for fingerspelling recognition of Chinese sign language

Hierarchical Palmprint Feature Extraction and Recognition Based on Multi-Wavelets and Complex Network

A HMM Based On-line Chinese Character Recognition System and Improved Training Algorithm

A Real-Time Large Vocabulary Continuous Recognition System For Chinese Sign Language

A Real-Time Large Vocabulary Recognition System for Chinese Sign Language.

A Sofm/Hmm System for Person-Independent Isolated Sign Language Recognition

Signer-Independent Sign Language Recognition Based On Sofm/Hmm

Optical Font Recognition Of Single Chinese Character

A Fine-Grained Visual Attention Approach for Fingerspelling Recognition in the Wild

Research on a Chinese Character Recognition Framework based on Multi-dimensional Image Information

A Computer System for Chinese Character Speech Input

Pre-merger Method of Chinese Online Handwritten Text Based on Classification

Online Chinese Handwriting Recognition with Time Sequence Information

A new system for Chinese sign language recognition