Abstract:Existing scene text recognition (STR) methods struggle to recognize challenging texts, especially for artistic and severely distorted characters. The limitation lies in the insufficient exploration of character morphologies, including the monotonousness of widely used synthetic training data and the sensitivity of the model to character morphologies. To address these issues, inspired by the human learning process of viewing and summarizing, we facilitate the contrastive learning-based STR framework in a self-motivated manner by leveraging synthetic and real unlabeled data without any human cost. In the viewing process, to compensate for the simplicity of synthetic data and enrich character morphology diversity, we propose an Online Generation Strategy to generate background-free samples with diverse character styles. By excluding background noise distractions, the model is encouraged to focus on character morphology and generalize the ability to recognize complex samples when trained with only simple synthetic data. To boost the summarizing process, we theoretically demonstrate the derivation error in the previous character contrastive loss, which mistakenly causes the sparsity in the intra-class distribution and exacerbates ambiguity on challenging samples. Therefore, a new Character Unidirectional Alignment Loss is proposed to correct this error and unify the representation of the same characters in all samples by aligning the character features in the student model with the reference features in the teacher model. Extensive experiment results show that our method achieves SOTA performance (94.7\% and 70.9\% average accuracy on common benchmarks and Union14M-Benchmark). Code will be available at <a class="link-external link-https" href="https://github.com/qqqyd/ViSu" rel="external noopener nofollow">this https URL</a>.

Group, Contrast and Recognize: A Self-supervised Method for Chinese Character Recognition.

GroupContrast: Semantic-aware Self-supervised Representation Learning for 3D Understanding

Align Yourself: Self-supervised Pre-training for Fine-grained Recognition via Saliency Alignment.

An approach for handwritten Chinese text recognition unifying character segmentation and recognition

Self-supervised Character-to-Character Distillation for Text Recognition

Chinese Character Recognition with Augmented Character Profile Matching

Saliency Guided Contrastive Learning on Scene Images

Mejigclu: more effective jigsaw clustering for unsupervised visual representation learning

A new approach for off-line handwritten Chinese character recognition using self-adaptive HMM

Boosting Semi-Supervised Scene Text Recognition via Viewing and Summarizing

Occluded Facial Expression Recognition Using Self-supervised Learning

Weakly-Supervised Text-driven Contrastive Learning for Facial Behavior Understanding

Chinese Text Recognition with A Pre-Trained CLIP-Like Model Through Image-IDS Aligning

Pursuing Knowledge Consistency: Supervised Hierarchical Contrastive Learning for Facial Action Unit Recognition

Cross-Domain Facial Expression Recognition via Contrastive Warm up and Complexity-Aware Self-Training

Recognition of Handwritten Chinese Text by Segmentation: A Segment-annotation-free Approach

Learning Spatial-Semantic Context with Fully Convolutional Recurrent Network for Online Handwritten Chinese Text Recognition

Similar Handwritten Chinese Character Discrimination by Weakly Supervised Learning

Recognition Confidence Analysis of Handwritten Chinese Character with CNN

A character image restoration method for unconstrained handwritten Chinese character recognition

Self-Supervised Visual Representations Learning by Contrastive Mask Prediction