Abstract:Recent advanced Table Structure Recognition (TSR) models adopt image-to-text solutions to parse table structure. These methods can be formulated as image caption problem, i.e. , input a single-table image and output table structure description in a specific text format, e.g. , HTML. With the impressive success of Transformer in text generation tasks, these methods use Transformer architecture to predict HTML table text in an autoregressive manner. However, tables always emerge with a large variety of shapes and sizes. Autoregressive models usually suffer from the error accumulation problem as the length of predicted text increases, which results in unsatisfactory performance for large tables. In this paper, we propose a novel image-to-text based TSR method that relieves error accumulation problems and improves performance noticeably. At the core of our method is a cascaded two-step decoder architecture with the former decoder predicting HTML table row tags non-autoregressively and the latter predicting HTML table cell tags of each row in a semi-autoregressive manner. Compared with existing methods that predict HTML text autoregressively, the superiority of our row-to-cell progressive table parsing is twofold: (i) it generates an HTML tag sequence with a vertical-and-horizontal two-step 'scanning', which better fits the inherent 2D structure of image data, (ii) it performs substantially better for large tables (long sequence prediction) since it alleviates error accumulation problem specific to autoregressive models. Extensive experiments demonstrate that our method achieves competitive performance on three public benchmarks.

Structure-Aware Pre-Training for Table-to-Text Generation

Towards Table-to-Text Generation with Pretrained Language Model: A Table Structure Understanding and Text Deliberating Approach

Unifying Structured Data as Graph for Data-to-Text Pre-Training

Table Pre-training: A Survey on Model Architectures, Pre-training Objectives, and Downstream Tasks

TableGPT: Few-shot Table-to-Text Generation with Table Structure Reconstruction and Content Matching

Table-to-Text Generation by Structure-Aware Seq2seq Learning

DeepStruct: Pretraining of Language Models for Structure Prediction.

Structure-aware Pre-training for Table Understanding with Tree-based Transformers

Structure-aware Pre-training for Table Understanding with Tree-based Transformers.

Generative Table Pre-training Empowers Models for Tabular Prediction

TUTA: Tree-based Transformers for Generally Structured Table Pre-training

Table-To-Text generation and pre-training with TabT5

Bridge the Gap between Language models and Tabular Understanding

TableVLM: Multi-modal Pre-training for Table Structure Recognition

Robust (Controlled) Table-to-Text Generation with Structure-Aware Equivariance Learning

Self-Supervised Pre-Training for Table Structure Recognition Transformer

UniTable: Towards a Unified Framework for Table Structure Recognition via Self-Supervised Pretraining

Enhancing Pre-trained Models with Text Structure Knowledge for Question Generation

A Sequence-to-Sequence&Set Model for Text-to-Table Generation

Divide Rows and Conquer Cells: Towards Structure Recognition for Large Tables.

Text-to-Table: A New Way of Information Extraction