Abstract:In recent years, tremendous efforts have been made on document image rectification, but existing advanced algorithms are limited to processing restricted document images, i.e., the input images must incorporate a complete document. Once the captured image merely involves a local text region, its rectification quality is degraded and unsatisfactory. Our previously proposed DocTr, a transformer-assisted network for document image rectification, also suffers from this limitation. In this work, we present DocTr++, a novel unified framework for document image rectification, without any restrictions on the input distorted images. Our major technical improvements can be concluded in three aspects. Firstly, we upgrade the original architecture by adopting a hierarchical encoder-decoder structure for multi-scale representation extraction and parsing. Secondly, we reformulate the pixel-wise mapping relationship between the unrestricted distorted document images and the distortion-free counterparts. The obtained data is used to train our DocTr++ for unrestricted document image rectification. Thirdly, we contribute a real-world test set and metrics applicable for evaluating the rectification quality. To our best knowledge, this is the first learning-based method for the rectification of unrestricted document images. Extensive experiments are conducted, and the results demonstrate the effectiveness and superiority of our method. We hope our DocTr++ will serve as a strong baseline for generic document image rectification, prompting the further advancement and application of learning-based algorithms. The source code and the proposed dataset are publicly available at https://github.com/fh2019ustc/DocTr-Plus.

Deep Transformer: A Framework for 2D Text Image Rectification from Planar Transformations

Effective Document Image Rectification via a Deep Learning Framework

Braille Recognition Using Deep Learning

Deep Unrestricted Document Image Rectification.

DeepText: A Unified Framework for Text Proposal Generation and Text Detection in Natural Images

Curved Text Detection in Scenic Images via Proposal-Free Panoptic Segmentation and Deep Learning

Focus-Enhanced Scene Text Recognition with Deformable Convolutions

Robust Scene Text Recognition with Automatic Rectification

Transformer text recognition with deep learning algorithm

A Scheme for Automatic Text Rectification in Real Scene Images.

Rethinking Irregular Scene Text Recognition

SPIN: Structure-Preserving Inner Offset Network for Scene Text Recognition

A Deep Learning Approach for Robust, Multi-oriented, and Curved Text Detection

Geometric Representation Learning for Document Image Rectification

Image rectangling network based on reparameterized transformer and assisted learning

Leveraging Systematic Knowledge of 2D Transformations

Robustly Recognizing Irregular Scene Text by Rectifying Principle Irregularities

DPTNet: A Dual-Path Transformer Architecture for Scene Text Detection

A Two-level Rectification Attention Network for Scene Text Recognition

Deep Learning-based Text-in-Image Watermarking