Abstract:Information Extraction (IE) from document images is challenging due to the high variability of layout formats. Deep models such as LayoutLM and BROS have been proposed to address this problem and have shown promising results. However, they still require a large amount of field-level annotations for training these models. Other approaches using rule-based methods have also been proposed based on the understanding of the layout and semantics of a form such as geometric position, or type of the fields, etc. In this work, we propose a novel approach, EIGEN (Expert-Informed Joint Learning aGgrEatioN), which combines rule-based methods with deep learning models using data programming approaches to circumvent the requirement of annotation of large amounts of training data. Specifically, EIGEN consolidates weak labels induced from multiple heuristics through generative models and use them along with a small number of annotated labels to jointly train a deep model. In our framework, we propose the use of labeling functions that include incorporating contextual information thus capturing the visual and language context of a word for accurate categorization. We empirically show that our EIGEN framework can significantly improve the performance of state-of-the-art deep models with the availability of very few labeled data instances. The source code is available at <a class="link-external link-https" href="https://github.com/ayushayush591/EIGEN-High-Fidelity-Extraction-Document-Images" rel="external noopener nofollow">this https URL</a>.

Information Extraction from Invoices: A Graph Neural Network Approach for Datasets with High Layout Variety

An Invoice Reading System Using a Graph Convolutional Network

An Overview of Data Extraction From Invoices

Graph-based Deep Generative Modelling for Document Layout Generation

An Iterative Graph Learning Convolution Network for Key Information Extraction Based on the Document Inductive Bias.

Robust Layout-aware IE for Visually Rich Documents with Pre-trained Language Models

Doc-GCN: Heterogeneous Graph Convolutional Networks for Document Layout Analysis

GraphRevisedIE: Multimodal Information Extraction with Graph-Revised Network

CUTIE: Learning to Understand Documents with Convolutional Universal Text Information Extractor

Key Information Extraction From Documents: Evaluation And Generator

A Deep Generative Model for Graph Layout

EIGEN: Expert-Informed Joint Learning Aggregation for High-Fidelity Information Extraction from Document Images

A Joint Learning Information Extraction Method Based on an Effective Inference Structure

Graph Convolution for Multimodal Information Extraction from Visually Rich Documents

Identifying Illicit Accounts in Large Scale E-payment Networks -- A Graph Representation Learning Approach

PARAGRAPH2GRAPH: A GNN-based framework for layout paragraph analysis

ExTTNet: A Deep Learning Algorithm for Extracting Table Texts from Invoice Images

Exploring the Use of Graph Neural Networks for Blockchain Transaction Analysis and Fraud Detection

Pick and Choose: A GNN-based Imbalanced Learning Approach for Fraud Detection

DocGraphLM: Documental Graph Language Model for Information Extraction

Table understanding in structured documents