Abstract:Restoring the original, flat appearance of a printed document from casual photographs of bent and wrinkled pages is a common everyday problem. In this paper we propose a novel method for grid-based single-image document unwarping. Our method performs geometric distortion correction via a fully convolutional deep neural network that learns to predict the 3D grid mesh of the document and the corresponding 2D unwarping grid in a dual-task fashion, implicitly encoding the coupling between the shape of a 3D piece of paper and its 2D image. In order to allow unwarping models to train on data that is more realistic in appearance than the commonly used synthetic Doc3D dataset, we create and publish our own dataset, called UVDoc, which combines pseudo-photorealistic document images with physically accurate 3D shape and unwarping function annotations. Our dataset is labeled with all the information necessary to train our unwarping network, without having to engineer separate loss functions that can deal with the lack of ground-truth typically found in document in the wild datasets. We perform an in-depth evaluation that demonstrates that with the inclusion of our novel pseudo-photorealistic dataset, our relatively small network architecture achieves state-of-the-art results on the DocUNet benchmark. We show that the pseudo-photorealistic nature of our UVDoc dataset allows for new and better evaluation methods, such as lighting-corrected MS-SSIM. We provide a novel benchmark dataset that facilitates such evaluations, and propose a metric that quantifies line straightness after unwarping. Our code, results and UVDoc dataset are available at <a class="link-external link-https" href="https://github.com/tanguymagne/UVDoc" rel="external noopener nofollow">this https URL</a>.

Arbitrary Warped Document Image Restoration Based on Segmentation and Thin-Plate Splines.

Restoring Warped Document Image Through Segmentation and Full Page Interpolation

Restoring Camera-Captured Distorted Document Images

Document Forgery Detection Using Distortion Mutation of Geometric Parameters in Characters

A Warped Document Image Mosaicing Method Based on Registration and TRS Transform

Printer Forensics Based on Page Document's Geometric Distortion

A method to restore Chinese warped document images based on binding characters and building curved lines

Adaptive dewarping of severely warped camera-captured document images based on document map generation

Restoring Chinese documents images based on text boundary lines

Image Processing For Virtual Recovery Of Murals

Document Image Segmentation Using Gabor Wavelet and Kernel-based Methods

DocTr: Document Image Transformer for Geometric Unwarping and Illumination Correction

Learning from Documents in the Wild to Improve Document Unwarping

A Document Rectification Approach Dealing with Both Perspective Distortion and Warping Based on Text Flow Curve Fitting

MataDoc: Margin and Text Aware Document Dewarping for Arbitrary Boundary

Layout-aware Single-image Document Flattening

Image Mosaicking for Oversized Documents with a Multi-Camera Rig

Fourier Document Restoration for Robust Document Dewarping and Recognition

UVDoc: Neural Grid-based Document Unwarping

Segmentation and Recognition for Historical Tibetan Document Images

Layout and Perspective Distortion Independent Recognition of Captured Chinese Document Image