Abstract:Handwritten Chinese text recognition (HCTR) is still a challenging and unsolved problem. The existing recognition methods are mainly categorized into two: explicit vs implicit segmentation-based methods. Explicit segmentation recognition methods use explicit character location information to train the recognizers. However, the widely used weakly supervised training strategy based on pseudo-label makes it difficult to get effective supervised training for difficult character samples. In contrast, the implicit segmentation recognition method use all transcript annotations for supervised training, but it is prone to misalignment problem due to the lack of explicit supervised information of character positions. To take advantage of the complementary nature of explicit and implicit segmentation approaches, we propose a new method, SegCTC, which better integrates these two approaches into a unified to be a more powerful recognizer. Specifically, we designed a hybrid Segmentation-based and Segmentation-free Feature Fusion Module (S $$^2$$ FFM) to better fuse the features of both explicit and implicit segmentation-based recognition branches. Moreover, a co-transcription strategy is also proposed to better combine the predictions from different branches. Experiments on four widely used benchmarks including CASIA-HWDB, ICDAR2013, SCUT-HCCDoc and MTHv2 show that our method achieves state-of-the-art performance for the HCTR task under different scenarios.

Automatic Detection of the Temporal Segmentation of Hand Movements in British English Cued Speech

Dynamic hand gesture recognition using hidden Markov models

Re-Synchronization Using the Hand Preceding Model for Multi-Modal Fusion in Automatic Continuous Cued Speech Recognition

A New Re-synchronization Method Based Multi-modal Fusion for Automatic Continuous Cued Speech Recognition

A Novel Interpretable and Generalizable Re-synchronization Model for Cued Speech based on a Multi-Cuer Corpus

Development of speechreading supplements based on automatic speech recognition

An Attention Self-supervised Contrastive Learning based Three-stage Model for Hand Shape Feature Representation in Cued Speech

Investigating the dynamics of hand and lips in French Cued Speech using attention mechanisms and CTC-based decoding

A Novel Resynchronization Procedure For Hand-Lips Fusion Applied To Continuous French Cued Speech Recognition

HMM-Based Recognizer with Segmentation-free Strategy for Unconstrained Chinese Handwritten Text

A Continuous Chinese Sign Language Recognition System

Modelling and Segmenting Subunits for Sign Language Recognition Based on Hand Motion Analysis

Chinese sign language recognition with adaptive HMM

Subunit Boundary Detection for Sign Language Recognition Using Spatio-temporal Modelling

A Threshold-based HMM-DTW Approach for Continuous Sign Language Recognition

Sign Language Recognition Based On Hmm/Ann/Dp

SegCTC: Offline Handwritten Chinese Text Recognition Via Better Fusion Between Explicit and Implicit Segmentation

An Adaptive Hand-Talking System for the Hearing Impaired

A Real-Time Large Vocabulary Continuous Recognition System For Chinese Sign Language

THE APPLICATION OF HMMs CONSISTING OF DIFFERENT NUMBER STATES IN CHINESE SIGN LANGUAGE RECOGNITION

A Vision-Based Sign Language Recognition System Using Tied-Mixture Density HMM