Abstract:Recognizing texts in images plays an important role in many applications, such as industrial intelligence, robot vision, automatic driving, command assistance, and scene understanding. Although great progress has been achieved in various fields, research on complex systems modeling using text recognition technology requires further attention. To address this, we propose a new end-to-end multi-task learning method, which includes a super-resolution branch (SRB) and a recognition branch. To effectively learn the semantic information of images, we utilize the feature pyramid network (FPN) to fuse high- and low-level semantic information. The feature map generated by FPN is then delivered separately to the super-resolution branch and the recognition branch. We introduce a novel super-resolution branch, the SRB based on the proposed dual attention mechanism (DAM), designed to enhance the capability of learning low-resolution text features. The DAM incorporates the residual channel attention to enhance channel dependencies and the character attention module to focus on context information. For the recognition branch, the feature map generated by FPN is fed into an RNN sequence module, and an attention-based decoder is constructed to predict the results. To address the issue of low-resolution text recognition in numerous Chinese scenes, we propose the Chinese super-resolution datasets instead of relying on traditional down-sampling techniques to generate training datasets. Experiments demonstrate that the proposed method performs robustly on low-resolution text images and achieves competitive results on benchmark datasets.

End-to-End Multilingual Text Recognition Based on Byte Modeling

Cross-Lingual Text Image Recognition Via Multi-Task Sequence to Sequence Learning.

Bytes are All You Need: End-to-End Multilingual Speech Recognition and Synthesis with Bytes

EMU: Effective Multi-Hot Encoding Net for Lightweight Scene Text Recognition with a Large Character Set.

Post-Processing Approach for Printed Chinese Character Recognition

A Unified Multilingual Handwriting Recognition System using multigrams sub-lexical units

Local Byte Fusion for Neural Machine Translation

Word-Level Representation From Bytes For Language Modeling

Integrating Multi-scale Contextualized Information for Byte-based Neural Machine Translation

A Multiplexed Network for End-to-End, Multilingual OCR

From Pixels to Tokens: Byte-Pair Encoding on Quantized Visual Modalities

Large Multilingual Models Pivot Zero-Shot Multimodal Learning across Languages

Optimizing Byte-level Representation for End-to-end ASR

TextMonkey: An OCR-Free Large Multimodal Model for Understanding Document

Multi-task Recurrent Model for True Multilingual Speech Recognition

On the Hidden Mystery of OCR in Large Multimodal Models

MTSTR: Multi-task learning for low-resolution scene text recognition via dual attention mechanism and its application in logistics industry

Training Multilingual Pre-trained Language Model with Byte-level Subwords

From Text to Pixel: Advancing Long-Context Understanding in MLLMs

Collaborative Multilingual Continuous Sign Language Recognition: A Unified Framework

A Multimodal Text Block Segmentation Framework for Photo Translation.