Abstract: Automatic document content processing is affected by artifacts caused by the shape of the paper, non-uniform and diverse color of lighting conditions. Fully-supervised methods on real data are impossible due to the large amount of data needed. Hence, the current state of the art deep learning models are trained on fully or partially synthetic images. However, document shadow or shading removal results still suffer because: (a) prior methods rely on uniformity of local color statistics, which limit their application on real-scenarios with complex document shapes and textures and; (b) synthetic or hybrid datasets with non-realistic, simulated lighting conditions are used to train the models. In this paper we tackle these problems with our two main contributions. First, a physically constrained learning-based method that directly estimates document reflectance based on intrinsic image formation which generalizes to challenging illumination conditions. Second, a new dataset that clearly improves previous synthetic ones, by adding a large range of realistic shading and diverse multi-illuminant conditions, uniquely customized to deal with documents in-the-wild. The proposed architecture works in a self-supervised manner where only the synthetic texture is used as a weak training signal (obviating the need for very costly ground truth with disentangled versions of shading and reflectance). The proposed approach leads to a significant generalization of document reflectance estimation in real scenes with challenging illumination. We extensively evaluate on the real benchmark datasets available for intrinsic image decomposition and document shadow removal tasks. Our reflectance estimation scheme, when used as a pre-processing step of an OCR pipeline, shows a 26% improvement of character error rate (CER), thus, proving the practical applicability.

Efficient Joint Rectification of Photometric and Geometric Distortions in Document Images.

Unified Optical Distortion Correction Method for Imaging Systems Using A Concise Geometrical Transformation Model

Effective Document Image Rectification via a Deep Learning Framework

DocTr: Document Image Transformer for Geometric Unwarping and Illumination Correction

Geometric Representation Learning for Document Image Rectification

DocScanner: Robust Document Image Rectification with Progressive Learning

Deep Unrestricted Document Image Rectification.

Layout-aware Single-image Document Flattening

An efficient deep learning-based framework for image distortion correction

Intrinsic Decomposition of Document Images In-the-Wild

DocMAE: Document Image Rectification via Self-supervised Representation Learning

Adaptive dewarping of severely warped camera-captured document images based on document map generation

ShaDocFormer: A Shadow-Attentive Threshold Detector With Cascaded Fusion Refiner for Document Shadow Removal

UDoc-GAN: Unpaired Document Illumination Correction with Background Light Prior.

DocUNet: Document Image Unwarping Via a Stacked U-Net

UVDoc: Neural Grid-based Document Unwarping

Restoring Camera-Captured Distorted Document Images

RecRecNet: Rectangling Rectified Wide-Angle Images by Thin-Plate Spline Model and DoF-based Curriculum Learning

A Warped Document Image Mosaicing Method Based on Registration and TRS Transform

Rethinking Supervision in Document Unwarping: A Self-consistent Flow-free Approach

DECDM: Document Enhancement using Cycle-Consistent Diffusion Models