Abstract:Single image rectification of document deformation is a challenging task. Although some recent deep learning-based methods have attempted to solve this problem, they cannot achieve satisfactory results when dealing with document images with complex deformations. In this article, we propose a new efficient framework for document flattening. Our main insight is that most layout primitives in a document have rectangular outline shapes, making unwarping local layout primitives essentially homogeneous with unwarping the entire document. The former task is clearly more straightforward to solve than the latter due to the more consistent texture and relatively smooth deformation. On this basis, we propose a layout-aware deep model working in a divide-and-conquer manner. First, we employ a transformer-based segmentation module to obtain the layout information of the input document. Then a new regression module is applied to predict the global and local UV maps. Finally, we design an effective merging algorithm to correct the global prediction with local details. Both quantitative and qualitative experimental results demonstrate that our framework achieves favorable performance against state-of-the-art methods. In addition, the current publicly available document flattening datasets have limited 3D paper shapes without layout annotation and also lack a general geometric correction metric. Therefore, we build a new large-scale synthetic dataset by utilizing a fully automatic rendering method to generate deformed documents with diverse shapes and exact layout segmentation labels. We also propose a new geometric correction metric based on our paired document UV maps. Code and dataset will be released at https://github.com/BunnySoCrazy/LA-DocFlatten .

Document Page Structure Learning For Fixed-Layout E-Books Using Conditional Random Fields

Logical Labeling of Fixed Layout PDF Documents Using Multiple Contexts

Contextual Modeling for Logical Labeling of PDF Documents

CNN Based Page Object Detection in Document Images

A Deep Learning-Based Formula Detection Method for Pdf Documents

Structure extraction from PDF-based book documents.

Graph-based Layout Analysis for PDF Documents

VSR: A Unified Framework for Document Layout Analysis Combining Vision, Semantics and Relations

VTLayout: Fusion of Visual and Text Features for Document Layout Analysis

Reflowing-driven Paragraph Recognition for Electronic Books in PDF

Image Layer Modeling for Complex Document Layout Generation.

Capturing Logical Structure of Visually Structured Documents with Multimodal Transition Parser

A Fusion Framework of Whitespace Smear Cutting and Swin Transformer for Document Layout Analysis

Are Layout-Infused Language Models Robust to Layout Distribution Shifts? A Case Study with Scientific Documents

DocStruct: A Multimodal Method to Extract Hierarchy Structure in Document for General Form Understanding

LayoutReader: Pre-training of Text and Layout for Reading Order Detection

Layout and Perspective Distortion Independent Recognition of Captured Chinese Document Image

Layout-aware Single-image Document Flattening

Joint Layout Analysis, Character Detection and Recognition for Historical Document Digitization

Synthetic document generator for annotation-free layout recognition

Document AI: A Comparative Study of Transformer-Based, Graph-Based Models, and Convolutional Neural Networks For Document Layout Analysis