Abstract:Table-based reasoning with large language models (LLMs) is a promising direction to tackle many table understanding tasks, such as table-based question answering and fact verification. Compared with generic reasoning, table-based reasoning requires the extraction of underlying semantics from both free-form questions and semi-structured tabular data. Chain-of-Thought and its similar approaches incorporate the reasoning chain in the form of textual context, but it is still an open question how to effectively leverage tabular data in the reasoning chain. We propose the Chain-of-Table framework, where tabular data is explicitly used in the reasoning chain as a proxy for intermediate thoughts. Specifically, we guide LLMs using in-context learning to iteratively generate operations and update the table to represent a tabular reasoning chain. LLMs can therefore dynamically plan the next operation based on the results of the previous ones. This continuous evolution of the table forms a chain, showing the reasoning process for a given tabular problem. The chain carries structured information of the intermediate results, enabling more accurate and reliable predictions. Chain-of-Table achieves new state-of-the-art performance on WikiTQ, FeTaQA, and TabFact benchmarks across multiple LLM choices.

What problem does this paper attempt to address?

The paper primarily addresses issues in the field of natural language processing, specifically focusing on the understanding and reasoning of tabular data. Specifically, the researchers propose a new framework called CHAIN-OF-TABLE, which aims to form a "table chain" by progressively manipulating tabular data to help large language models (LLMs) better understand and solve table-based tasks, such as table question answering and fact verification. The paper points out that traditional table understanding methods have some limitations, such as the inability to effectively utilize information within the table structure for reasoning. Therefore, the researchers propose a new approach by defining a series of table operations (such as adding columns, selecting rows, etc.) and dynamically generating these operations under the guidance of a given question, thereby gradually constructing a table chain that includes intermediate results. This approach allows LLMs to progressively modify and update the table according to the current question's requirements, making it easier to arrive at the correct answer. The method was experimentally validated on several benchmark datasets, including WikiTQ, FeTaQA, and TabFact. The results show that CHAIN-OF-TABLE outperforms several existing methods on these tasks, including text-based Chain-of-Thought reasoning methods and program-assisted methods (such as generating SQL queries or Python scripts). Additionally, the paper analyzes the impact of different lengths of operation chains on performance and discusses the method's performance on input tables of varying sizes. In summary, the goal of this paper is to introduce a new multi-step reasoning framework in table understanding tasks to improve the accuracy and reliability of large language models in solving complex table problems.

Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understanding

Large Language Models are few(1)-shot Table Reasoners

Rethinking Tabular Data Understanding with Large Language Models

Tree-of-Table: Unleashing the Power of LLMs for Enhanced Large-Scale Table Understanding

Seek and Solve Reasoning for Table Question Answering

ChainLM: Empowering Large Language Models with Improved Chain-of-Thought Prompting

Reasoning over Hybrid Chain for Table-and-Text Open Domain Question Answering

FLEXTAF: Enhancing Table Reasoning with Flexible Tabular Formats

Reasoning over Hybrid Chain for Table-and-Text Open Domain QA

TabSQLify: Enhancing Reasoning Capabilities of LLMs Through Table Decomposition

Bi-Chainer: Automated Large Language Models Reasoning with Bidirectional Chaining

Chain-of-Thought Hub: A Continuous Effort to Measure Large Language Models' Reasoning Performance

Towards Faithful Chain-of-Thought: Large Language Models are Bridging Reasoners

Enhancing Temporal Understanding in LLMs for Semi-structured Tables

A Survey of Table Reasoning with Large Language Models

TART: An Open-Source Tool-Augmented Framework for Explainable Table-based Reasoning

Chain of Logic: Rule-Based Reasoning with Large Language Models

ChatCoT: Tool-Augmented Chain-of-Thought Reasoning on Chat-based Large Language Models

Table Meets LLM: Can Large Language Models Understand Structured Table Data? A Benchmark and Empirical Study

Aggregation of Reasoning: A Hierarchical Framework for Enhancing Answer Selection in Large Language Models