TAP4LLM: Table Provider on Sampling, Augmenting, and Packing Semi-structured Data for Large Language Model Reasoning

Yuan Sui,Jiaru Zou,Mengyu Zhou,Xinyi He,Lun Du,Shi Han,Dongmei Zhang

2024-10-10

Abstract:Table reasoning tasks have shown remarkable progress with the development of large language models (LLMs), which involve interpreting and drawing conclusions from tabular data based on natural language (NL) questions. Existing solutions mainly tested on smaller tables face scalability issues and struggle with complex queries due to incomplete or dispersed data across different table sections. To alleviate these challenges, we propose TAP4LLM as a versatile pre-processor suite for leveraging LLMs in table-based tasks effectively. It covers several distinct components: (1) table sampling to decompose large tables into manageable sub-tables based on query semantics, (2) table augmentation to enhance tables with additional knowledge from external sources or models, and (3) table packing & serialization to convert tables into various formats suitable for LLMs' understanding. In each module, we design and compare several common methods under various usage scenarios, aiming to shed light on the best practices for leveraging LLMs for table-reasoning tasks. Our experiments show that our method improves LLMs' reasoning capabilities in various tabular tasks and enhances the interaction between LLMs and tabular data by employing effective pre-processing.

Computation and Language,Artificial Intelligence

What problem does this paper attempt to address?

The paper attempts to address the challenges faced by large language models (LLMs) when processing tabular data, particularly in terms of scalability and accuracy when dealing with large-scale tables and complex queries. Existing solutions are primarily tested on smaller tables and experience performance degradation when faced with large tables or complex queries that require data collection from different parts of the table. To mitigate these challenges, the authors propose the TAP4LLM framework, a versatile preprocessor suite for effectively utilizing large language models in table tasks.

Specifically, TAP4LLM includes three core modules:
1. **Table Sampling**: Decomposes large tables into manageable sub-tables based on query semantics.
2. **Table Augmentation**: Adds additional knowledge to the table from external sources or models.
3. **Table Packing & Serialization**: Converts tables into various formats suitable for LLMs' understanding while controlling token allocation.

The goal of the paper is to explore how to best utilize large language models for table reasoning tasks through the design and comparison of these three modules, thereby improving LLMs' reasoning capabilities and interaction with tabular data across various table tasks.