Abstract:Due to the concise and structured nature of tables, the knowledge contained therein may be incomplete or missing, posing a significant challenge for table question answering (TableQA) and data analysis systems. Most existing datasets either fail to address the issue of external knowledge in TableQA or only utilize unstructured text as supplementary information for tables. In this paper, we propose to use a knowledge base (KB) as the external knowledge source for TableQA and construct a dataset KET-QA with fine-grained gold evidence annotation. Each table in the dataset corresponds to a sub-graph of the entire KB, and every question requires the integration of information from both the table and the sub-graph to be answered. To extract pertinent information from the vast knowledge sub-graph and apply it to TableQA, we design a retriever-reasoner structured pipeline model. Experimental results demonstrate that our model consistently achieves remarkable relative performance improvements ranging from 1.9 to 6.5 times and absolute improvements of 11.66% to 44.64% on EM scores across three distinct settings (fine-tuning, zero-shot, and few-shot), in comparison with solely relying on table information in the traditional TableQA manner. However, even the best model achieves a 60.23% EM score, which still lags behind the human-level performance, highlighting the challenging nature of KET-QA for the question-answering community. We also provide a human evaluation of error cases to analyze further the aspects in which the model can be improved. Project page:

What is in the KGQA Benchmark Datasets? Survey on Challenges in Datasets for Question Answering on Knowledge Graphs

What is in the KGQA Benchmark Datasets? Survey on Challenges in Datasets for Question Answering on Knowledge Graphs

Spider4SPARQL: A Complex Benchmark for Evaluating Knowledge Graph Question Answering Systems

Knowledge Graph Question Answering Leaderboard: A Community Resource to Prevent a Replication Crisis

Knowledge Graph Question Answering Datasets and Their Generalizability: Are They Enough for Future Research?

A Universal Question-Answering Platform for Knowledge Graphs

Would You Ask it that Way? Measuring and Improving Question Naturalness for Knowledge Graph Question Answering

Modern Question Answering Datasets and Benchmarks: A Survey

Do I have the Knowledge to Answer? Investigating Answerability of Knowledge Base Questions

Leveraging Knowledge Graph for Open-Domain Question Answering

CR-LT-KGQA: A Knowledge Graph Question Answering Dataset Requiring Commonsense Reasoning and Long-Tail Knowledge

A Survey on Complex Knowledge Base Question Answering: Methods, Challenges and Solutions

Core techniques of question answering systems over knowledge bases: a survey

KET-QA: A Dataset for Knowledge Enhanced Table Question Answering

Can Knowledge Graphs Make Large Language Models More Trustworthy? An Empirical Study over Open-ended Question Answering

Formal Query Generation for Question Answering over Knowledge Bases

No One is Perfect: Analysing the Performance of Question Answering Components over the DBpedia Knowledge Graph

Introduction to neural network‐based question answering over knowledge graphs

ChatGPT versus Traditional Question Answering for Knowledge Graphs: Current Status and Future Directions Towards Knowledge Graph Chatbots