Abstract:Text-to-Table aims to generate structured tables to convey the key information from unstructured documents. Existing text-to-table datasets are typically oriented English, limiting the research in non-English languages. Meanwhile, the emergence of large language models (LLMs) has shown great success as general task solvers in multi-lingual settings (e.g., ChatGPT), theoretically enabling text-to-table in other languages. In this paper, we propose a Chinese text-to-table dataset, CT-Eval, to benchmark LLMs on this task. Our preliminary analysis of English text-to-table datasets highlights two key factors for dataset construction: data diversity and data hallucination. Inspired by this, the CT-Eval dataset selects a popular Chinese multidisciplinary online encyclopedia as the source and covers 28 domains to ensure data diversity. To minimize data hallucination, we first train an LLM to judge and filter out the task samples with hallucination, then employ human annotators to clean the hallucinations in the validation and testing sets. After this process, CT-Eval contains 88.6K task samples. Using CT-Eval, we evaluate the performance of open-source and closed-source LLMs. Our results reveal that zero-shot LLMs (including GPT-4) still have a significant performance gap compared with human judgment. Furthermore, after fine-tuning, open-source LLMs can significantly improve their text-to-table ability, outperforming GPT-4 by a large margin. In short, CT-Eval not only helps researchers evaluate and quickly understand the Chinese text-to-table ability of existing LLMs but also serves as a valuable resource to significantly improve the text-to-table performance of LLMs.

Automatic Table Boundary Detection and Performance Evaluation in Fixed-Layout Documents

Design and Realization of Automatic Performance Evaluation of Table Processing

Dataset, Ground-Truth and Performance Metrics for Table Detection Evaluation

An Automatic Performance Evaluation Method for Document Page Segmentation

Table detection in online ink notes.

DOCUMENT IMAGE ADAPTIVE MOBILE READING TECHNOLOGY WITHOUT FONT SUPPORT

Ground-Truth and Performance Evaluation for Page Layout Analysis of Born-Digital Documents

Table Detection from Plain Text Using Machine Learning and Document Structure

Analysis of Book Documents' Table of Content Based on Clustering

An OpenCV-based Framework for Table Information Extraction

TDeLTA: A Light-weight and Robust Table Detection Method based on Learning Text Arrangement

A Novel Encoder-Decoder Architecture for Table Border Segmentation of Scanned Documents.

A Mixed Approach to Auto-Detection of Page Body

A Table of Content Recognition Method of Book Documents Based on Clustering Techniques

A YOLO-Based Table Detection Method

DESIGNING AND IMPLEMENTING ELECTRONIC PAPER SCREEN BASED DOCUMENT LAYOUT RECONSTRUCTION

Detecting Table Region in PDF Documents Using Distant Supervision

Automatic Recognition System of Tabular Document and Its Application

Analysis, Understanding and Representation of Chinese Newspaper with Complex Layout

A Method of Evaluating Table Segmentation Results Based on a Table Image Ground Truther

CT-Eval: Benchmarking Chinese Text-to-Table Performance in Large Language Models