Abstract:The number of published PDF documents in both the academic and commercial world has increased exponentially in recent decades. There is a growing need to make their rich content discoverable to information retrieval tools. Achieving high-quality semantic searches demands that a document's structural components such as title, section headers, paragraphs, (nested) lists, tables and figures (including their captions) are properly identified. Unfortunately, the PDF format is known to not conserve such structural information because it simply represents a document as a stream of low-level printing commands, in which one or more characters are placed in a bounding box with a particular styling. In this paper, we present a novel approach to document structure recovery in PDF using recurrent neural networks to process the low-level PDF data representation directly, instead of relying on a visual re-interpretation of the rendered PDF page, as has been proposed in previous literature. We demonstrate how a sequence of PDF printing commands can be used as input into a neural network and how the network can learn to classify each printing command according to its structural function in the page. This approach has three advantages: First, it can distinguish among more fine-grained labels (typically 10-20 labels as opposed to 1-5 with visual methods), which results in a more accurate and detailed document structure resolution. Second, it can take into account the text flow across pages more naturally compared to visual methods because it can concatenate the printing commands of sequential pages. Last, our proposed method needs less memory and it is computationally less expensive than visual methods. This allows us to deploy such models in production environments at a much lower cost. Through extensive architectural search in combination with advanced feature engineering, we were able to implement a model that yields a weighted average F1 score of 97% across 17 distinct structural labels. The best model we achieved is currently served in production environments on our Corpus Conversion Service (CCS), which was presented at KDD18. This model enhances the capabilities of CCS significantly, as it eliminates the need for human annotated label ground-truth for every unseen document layout. This proved particularly useful when applied to a huge corpus of PDF articles related to COVID-19.

Intervention of light convolutional neural network in document survey form processing

Fast and robust questionnaire recognition on mobile device

Handwritten Digit Recognition Based on Corner Detection and Convolutional Neural Network

Convolutional Neural Network Based Intelligent Handwritten Document Recognition

A Fast Fully Octave Convolutional Neural Network for Document Image Segmentation

Implementation of OCR using Convolutional Neural Network (CNN): A Survey

A Survey on Deep learning based Document Image Enhancement

Improvement of color image analysis using a hybrid artificial intelligence algorithm

Handwritten Digit Recognition using Image Preprocessing and CNN

Light-weight Document Image Cleanup using Perceptual Loss

Study Of The Application Of Deep Convolutional Neural Networks (Cnns) In Processing Sensor Data And Biomedical Images

AI-Driven Smartphone Solution for Digitizing Rapid Diagnostic Test Kits and Enhancing Accessibility for the Visually Impaired

Convolutional Neural Network Based Real Time Arabic Speech Recognition to Arabic Braille for Hearing and Visually Impaired

Assessing the Feasibility of Processing a Paper-based Multilingual Social Needs Screening Questionnaire Using Artificial Intelligence

Performance analysis of hybrid deep learning framework using a vision transformer and convolutional neural network for handwritten digit recognition

Visual question answering on blood smear images using convolutional block attention module powered object detection

Improved Handwritten Digit Recognition Using Convolutional Neural Networks (CNN)

Robust PDF Document Conversion using Recurrent Neural Networks

Handwritten Text Recognition Using Convolutional Neural Network

An offline English optical character recognition and NER using LSTM and adaptive neuro-fuzzy inference system

Deep Learning Approach Combining Lightweight CNN Architecture with Transfer Learning: An Automatic Approach for the Detection and Recognition of Bangladeshi Banknotes