Abstract:In this paper we study the task of document layout recognition for digital documents, requiring that the model should detect the exact physical object region without missing any text or containing any redundant text outside objects. It is the vital step to support high-quality information extraction, table understanding and knowledge base construction over the documents from various vertical domains (e.g. financial, legal, and government fields). Here, we consider digital documents, where characters and graphic elements are given with their exact texts, positions inside document pages, compared with image documents. Towards document layout recognition with pinpoint accuracy, we consider this problem as a document panoptic segmentation task, that each token in the document page must be assigned a class label and an instance id. Considering that two predicted objects may intersect under traditional visual panoptic segmentation method, like Mask R-CNN, however, document objects never intersect because most document pages follow manhattan layout. Therefore, we propose a novel framework, named document panoptic segmentation (DPS) model. It first splits the document page into column regions and groups tokens into line regions, then extracts the textual and visual features, and finally assigns class label and instance id to each line region. Additionally, we propose a novel metric based on the intersection over union (IoU) between the tokens contained in predicted and the ground-truth object, which is more suitable than metric based on the area IoU between predicted and the ground-truth bounding box. Finally, the empirical experiments based on PubLayNet, ArXiv and Financial datasets show that the proposed DPS model obtains 0.8833, 0.9205 and 0.8530 mAP scores on three datasets. The proposed model obtains great improvement on mAP score compared with Faster R-CNN and Mask R-CNN models.

Towards Accurate Panel Detection in Manga: A Combined Effort of CNN and Heuristics.

CNN Based Page Object Detection in Document Images

A tree conditional random field model for panel detection in comic images

Accurate Fish Detection under Marine Background Noise Based on the Retinex Enhancement Algorithm and CNN

Comic Image ' Edge Point Detection Chaining Procedure Edge Segment Detection

An End-to-End Quadrilateral Regression Network for Comic Panel Extraction.

Automatic Comic Page Image Understanding Based on Edge Segment Analysis

A Faster R-CNN Based Method for Comic Characters Face Detection

Towards Document Panoptic Segmentation with Pinpoint Accuracy: Method and Evaluation

Screentone-Aware Manga Super-Resolution Using DeepLearning

Detection Method of Photovoltaic Panel Defect Based on Improved Mask R-CNN

Facial Landmark Detection for Manga Images

Progressive Deep Feature Learning for Manga Character Recognition Via Unlabeled Training Data

Automatic Comic Page Segmentation Based on Polygon Detection.

Comic Storyboard Extraction Via Edge Segment Analysis

Efficient and Accurate Arbitrary-Shaped Text Detection with Pixel Aggregation Network

Unconstrained Text Detection in Manga: a New Dataset and Baseline

PageNet: Towards End-to-End Weakly Supervised Page-Level Handwritten Chinese Text Recognition

Automating Manga Character Analysis: A Robust Deep Vision-Transformer Approach to Facial Landmark Detection

Unsupervised Manga Character Re-identification via Face-body and Spatial-temporal Associated Clustering

Exploiting Aliasing for Manga Restoration