Abstract:The transformer-based encoder-decoder framework is becoming popular in scene text recognition, largely because it naturally integrates recognition clues from both visual and semantic domains. However, recent studies show that the two kinds of clues are not always well registered and therefore, feature and character might be misaligned in difficult text (e.g., with a rare shape). As a result, constraints such as character position are introduced to alleviate this problem. Despite certain success, visual and semantic are still separately modeled and they are merely loosely associated. In this paper, we propose a novel module called multi-domain character distance perception (MDCDP) to establish a visually and semantically related position embedding. MDCDP uses the position embedding to query both visual and semantic features following the cross-attention mechanism. The two kinds of clues are fused into the position branch, generating a content-aware embedding that well perceives character spacing and orientation variants, character semantic affinities, and clues tying the two kinds of information. They are summarized as the multi-domain character distance. We develop CDistNet that stacks multiple MDCDPs to guide a gradually precise distance modeling. Thus, the feature-character alignment is well build even though various recognition difficulties are presented. We verify CDistNet on ten challenging public datasets and two series of augmented datasets created by ourselves. The experiments demonstrate that CDistNet performs highly competitively. It not only ranks top-tier in standard benchmarks, but also outperforms recent popular methods by obvious margins on real and augmented datasets presenting severe text deformation, poor linguistic support, and rare character layouts. In addition, the visualization shows that CDistNet achieves proper information utilization in both visual and semantic domains. Our code is available at https://github.com/simplify23/CDistNet .

Constrained Relation Network For Character Detection In Scene Images

Character Region Awareness for Text Detection

Scene Text Recognition from Two-Dimensional Perspective

Focus on the Whole Character: Discriminative Character Modeling for Scene Text Recognition

CRNet: A Center-aware Representation for Detecting Text of Arbitrary Shapes

ContourNet: Taking a Further Step Toward Accurate Arbitrary-shaped Scene Text Detection.

Character Region Awareness Network for Scene Text Recognition

CORE-Text: Improving Scene Text Detection with Contrastive Relational Reasoning

Conceptual Text Region Network: Cognition-Inspired Accurate Scene Text Detection

R-Net: A Relationship Network for Efficient and Accurate Scene Text Detection

A Text-Context-Aware CNN Network for Multi-oriented and Multi-language Scene Text Detection.

Detecting Text in the Wild with Deep Character Embedding Network

DSRN: A Deep Scale Relationship Network for Scene Text Detection.

Robust Scene Text Detection for Multi-script Languages Using Deep Learning.

Attentive Relational Networks for Mapping Images to Scene Graphs

LRANet: Towards Accurate and Efficient Scene Text Detection with Low-Rank Approximation Network

CPN: Complementary Proposal Network for Unconstrained Text Detection

CDistNet: Perceiving Multi-Domain Character Distance for Robust Text Recognition.

Correlation Propagation Networks for Scene Text Detection

Deep Relational Reasoning Graph Network for Arbitrary Shape Text Detection

Scene Text Recognition with Sliding Convolutional Character Models