Abstract:Due to the enormous technical challenges and wide range of applications, scene text recognition (STR) has been an active research topic in computer vision for years. To tackle this tough problem, numerous innovative methods have been successively proposed, and incorporating linguistic knowledge into STR models has recently become a prominent trend. In this work, we first draw inspiration from the recent progress in Vision Transformer (ViT) to construct a conceptually simple yet functionally powerful vision STR model, which is built upon ViT and a tailored Adaptive Addressing and Aggregation (A$^3$) module. It already outperforms most previous state-of-the-art models for scene text recognition, including both pure vision models and language-augmented methods. To integrate linguistic knowledge, we further propose a Multi-Granularity Prediction strategy to inject information from the language modality into the model in an implicit way, \ie, subword representations (BPE and WordPiece) widely used in NLP are introduced into the output space, in addition to the conventional character level representation, while no independent language model (LM) is adopted. To produce the final recognition results, two strategies for effectively fusing the multi-granularity predictions are devised. The resultant algorithm (termed MGP-STR) is able to push the performance envelope of STR to an even higher level. Specifically, MGP-STR achieves an average recognition accuracy of $94\%$ on standard benchmarks for scene text recognition. Moreover, it also achieves state-of-the-art results on widely-used handwritten benchmarks as well as more challenging scene text datasets, demonstrating the generality of the proposed MGP-STR algorithm. The source code and models will be available at: \url{<a class="link-external link-https" href="https://github.com/AlibabaResearch/AdvancedLiterateMachinery/tree/main/OCR/MGP-STR" rel="external noopener nofollow">this https URL</a>}.

A Feasible Framework for Arbitrary-Shaped Scene Text Recognition

Scene Text Recognition Via Dual-path Network with Shape-driven Attention Alignment.

Exploring Font-independent Features for Scene Text Recognition

Flexible scene text recognition based on dual attention mechanism

Enhancing Scene Text Recognition by Strengthening Attention Alignment

Arbitrary Shape Scene Text Detection with Adaptive Text Region Representation

Scene Text Recognition from Two-Dimensional Perspective

An extended attention mechanism for scene text recognition

ASTS: A Unified Framework for Arbitrary Shape Text Spotting.

Towards Accurate Alignment and Sufficient Context in Scene Text Recognition

Focus on the Whole Character: Discriminative Character Modeling for Scene Text Recognition

Learning and Fusing Multi-Scale Representations for Accurate Arbitrary-Shaped Scene Text Recognition.

Orientation-Independent Chinese Text Recognition in Scene Images

Multi-granularity Deep Local Representations for Irregular Scene Text Recognition

Rethinking Irregular Scene Text Recognition

Multi-Granularity Prediction for Scene Text Recognition

Hierarchical Refined Attention for Scene Text Recognition.

Focus-Enhanced Scene Text Recognition with Deformable Convolutions

Character Region Awareness Network for Scene Text Recognition

Robust Scene Text Recognition with Automatic Rectification

Multi-Granularity Prediction with Learnable Fusion for Scene Text Recognition