Abstract:Single image rectification of document deformation is a challenging task. Although some recent deep learning-based methods have attempted to solve this problem, they cannot achieve satisfactory results when dealing with document images with complex deformations. In this article, we propose a new efficient framework for document flattening. Our main insight is that most layout primitives in a document have rectangular outline shapes, making unwarping local layout primitives essentially homogeneous with unwarping the entire document. The former task is clearly more straightforward to solve than the latter due to the more consistent texture and relatively smooth deformation. On this basis, we propose a layout-aware deep model working in a divide-and-conquer manner. First, we employ a transformer-based segmentation module to obtain the layout information of the input document. Then a new regression module is applied to predict the global and local UV maps. Finally, we design an effective merging algorithm to correct the global prediction with local details. Both quantitative and qualitative experimental results demonstrate that our framework achieves favorable performance against state-of-the-art methods. In addition, the current publicly available document flattening datasets have limited 3D paper shapes without layout annotation and also lack a general geometric correction metric. Therefore, we build a new large-scale synthetic dataset by utilizing a fully automatic rendering method to generate deformed documents with diverse shapes and exact layout segmentation labels. We also propose a new geometric correction metric based on our paired document UV maps. Code and dataset will be released at https://github.com/BunnySoCrazy/LA-DocFlatten .

Image Layer Modeling for Complex Document Layout Generation.

DocLLM: A layout-aware generative language model for multimodal document understanding

DLAFormer: An End-to-End Transformer For Document Layout Analysis

A Graphical Approach to Document Layout Analysis

LayoutLM: Pre-training of Text and Layout for Document Image Understanding

A Fusion Framework of Whitespace Smear Cutting and Swin Transformer for Document Layout Analysis

LayoutLLM: Layout Instruction Tuning with Large Language Models for Document Understanding

VTLayout: Fusion of Visual and Text Features for Document Layout Analysis

Cross-Domain Document Layout Analysis Using Document Style Guide

Layout-aware Single-image Document Flattening

UnSupDLA: Towards Unsupervised Document Layout Analysis

Synthetic document generator for annotation-free layout recognition

LayoutLLM: Large Language Model Instruction Tuning for Visually Rich Document Understanding

LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking

Unified HMM-based Layout Analysis Framework and Algorithm

A Large Dataset of Historical Japanese Documents with Complex Layouts

Image Synthesis from Layout with Locality-Aware Mask Adaption

M$^{6}$Doc: A Large-Scale Multi-Format, Multi-Type, Multi-Layout, Multi-Language, Multi-Annotation Category Dataset for Modern Document Layout Analysis

Amodal Layout Completion in Complex Outdoor Scenes.

A Hybrid Approach for Document Layout Analysis in Document images