Abstract:Restoring the original, flat appearance of a printed document from casual photographs of bent and wrinkled pages is a common everyday problem. In this paper we propose a novel method for grid-based single-image document unwarping. Our method performs geometric distortion correction via a fully convolutional deep neural network that learns to predict the 3D grid mesh of the document and the corresponding 2D unwarping grid in a dual-task fashion, implicitly encoding the coupling between the shape of a 3D piece of paper and its 2D image. In order to allow unwarping models to train on data that is more realistic in appearance than the commonly used synthetic Doc3D dataset, we create and publish our own dataset, called UVDoc, which combines pseudo-photorealistic document images with physically accurate 3D shape and unwarping function annotations. Our dataset is labeled with all the information necessary to train our unwarping network, without having to engineer separate loss functions that can deal with the lack of ground-truth typically found in document in the wild datasets. We perform an in-depth evaluation that demonstrates that with the inclusion of our novel pseudo-photorealistic dataset, our relatively small network architecture achieves state-of-the-art results on the DocUNet benchmark. We show that the pseudo-photorealistic nature of our UVDoc dataset allows for new and better evaluation methods, such as lighting-corrected MS-SSIM. We provide a novel benchmark dataset that facilitates such evaluations, and propose a metric that quantifies line straightness after unwarping. Our code, results and UVDoc dataset are available at <a class="link-external link-https" href="https://github.com/tanguymagne/UVDoc" rel="external noopener nofollow">this https URL</a>.

Layout-aware Single-image Document Flattening

UVDoc: Neural Grid-based Document Unwarping

Learning from Documents in the Wild to Improve Document Unwarping

Effective Document Image Rectification via a Deep Learning Framework

DocUNet: Document Image Unwarping Via a Stacked U-Net

DewarpNet: Single-Image Document Unwarping with Stacked 3D and 2D Regression Networks.

Image Layer Modeling for Complex Document Layout Generation.

Efficient Joint Rectification of Photometric and Geometric Distortions in Document Images.

A Fusion Framework of Whitespace Smear Cutting and Swin Transformer for Document Layout Analysis

Restoring Camera-Captured Distorted Document Images

Deep Unrestricted Document Image Rectification.

Rethinking Supervision in Document Unwarping: A Self-consistent Flow-free Approach

DLAFormer: An End-to-End Transformer For Document Layout Analysis

MataDoc: Margin and Text Aware Document Dewarping for Arbitrary Boundary

LayoutLM: Pre-training of Text and Layout for Document Image Understanding

A Document Rectification Approach Dealing with Both Perspective Distortion and Warping Based on Text Flow Curve Fitting

Unfolder: Fast localization and image rectification of a document with a crease from folding in half

Image Mosaicking for Oversized Documents with a Multi-Camera Rig

DocTr: Document Image Transformer for Geometric Unwarping and Illumination Correction

Geometric Representation Learning for Document Image Rectification

A Hybrid Approach for Document Layout Analysis in Document images