Abstract:In this paper we study the task of document layout recognition for digital documents, requiring that the model should detect the exact physical object region without missing any text or containing any redundant text outside objects. It is the vital step to support high-quality information extraction, table understanding and knowledge base construction over the documents from various vertical domains (e.g. financial, legal, and government fields). Here, we consider digital documents, where characters and graphic elements are given with their exact texts, positions inside document pages, compared with image documents. Towards document layout recognition with pinpoint accuracy, we consider this problem as a document panoptic segmentation task, that each token in the document page must be assigned a class label and an instance id. Considering that two predicted objects may intersect under traditional visual panoptic segmentation method, like Mask R-CNN, however, document objects never intersect because most document pages follow manhattan layout. Therefore, we propose a novel framework, named document panoptic segmentation (DPS) model. It first splits the document page into column regions and groups tokens into line regions, then extracts the textual and visual features, and finally assigns class label and instance id to each line region. Additionally, we propose a novel metric based on the intersection over union (IoU) between the tokens contained in predicted and the ground-truth object, which is more suitable than metric based on the area IoU between predicted and the ground-truth bounding box. Finally, the empirical experiments based on PubLayNet, ArXiv and Financial datasets show that the proposed DPS model obtains 0.8833, 0.9205 and 0.8530 mAP scores on three datasets. The proposed model obtains great improvement on mAP score compared with Faster R-CNN and Mask R-CNN models.

DeepLayout: A Semantic Segmentation Approach to Page Layout Analysis

Deep Learning Based Semantic Page Segmentation of Document Images in Chinese and English

WeLayout: WeChat Layout Analysis System for the ICDAR 2023 Competition on Robust Layout Segmentation in Corporate Documents

DCT-CompSegNet: fast layout segmentation in DCT compressed JPEG document images using deep feature learning

Beyond document object detection: instance-level segmentation of complex layouts

Unsupervised Deep Learning for Handwritten Page Segmentation

LayoutLM: Pre-training of Text and Layout for Document Image Understanding

Image Layer Modeling for Complex Document Layout Generation.

A Graphical Approach to Document Layout Analysis

LayoutReader: Pre-training of Text and Layout for Reading Order Detection

VSR: A Unified Framework for Document Layout Analysis Combining Vision, Semantics and Relations

Synthetic document generator for annotation-free layout recognition

A Hybrid Approach for Document Layout Analysis in Document images

XYLayoutLM: Towards Layout-Aware Multimodal Networks for Visually-Rich Document Understanding

Towards Document Panoptic Segmentation with Pinpoint Accuracy: Method and Evaluation

A Fusion Framework of Whitespace Smear Cutting and Swin Transformer for Document Layout Analysis

VTLayout: Fusion of Visual and Text Features for Document Layout Analysis

ReLayout: Towards Real-World Document Understanding via Layout-enhanced Pre-training

HSCA-Net: A Hybrid Spatial-Channel Attention Network in Multi-Scale Feature Pyramid for Document Layout Analysis

PLACE: Adaptive Layout-Semantic Fusion for Semantic Image Synthesis

UnSupDLA: Towards Unsupervised Document Layout Analysis