Abstract:Knowledge base question answering (KBQA) is a critical yet challenging task due to the vast number of entities within knowledge bases and the diversity of natural language questions posed by users. Unfortunately, the performance of most KBQA models tends to decline significantly in real-world scenarios where high-quality annotated data is insufficient. To mitigate the burden associated with manual annotation, we introduce FlexKBQA by utilizing Large Language Models (LLMs) as program translators for addressing the challenges inherent in the few-shot KBQA task. Specifically, FlexKBQA leverages automated algorithms to sample diverse programs, such as SPARQL queries, from the knowledge base, which are subsequently converted into natural language questions via LLMs. This synthetic dataset facilitates training a specialized lightweight model for the KB. Additionally, to reduce the barriers of distribution shift between synthetic data and real user questions, FlexKBQA introduces an executionguided self-training method to iterative leverage unlabeled user questions. Furthermore, we explore harnessing the inherent reasoning capability of LLMs to enhance the entire framework. Consequently, FlexKBQA delivers substantial flexibility, encompassing data annotation, deployment, and being domain agnostic. Through extensive experiments on GrailQA, WebQSP, and KQA Pro, we observe that under the few-shot even the more challenging zero-shot scenarios, FlexKBQA achieves impressive results with a few annotations, surpassing all previous baselines and even approaching the performance of supervised models, achieving a remarkable 93% performance relative to the fully-supervised models. We posit that FlexKBQA represents a significant advancement towards exploring better integration of large and lightweight models. The code is open-sourced.

Few-Shot Multihop Question Answering over Knowledge Base

Few-shot Multi-hop Question Answering over Knowledge Base

Few-shot In-context Learning for Knowledge Base Question Answering

BB-KBQA: BERT-Based Knowledge Base Question Answering

FlexKBQA: A Flexible LLM-Powered Framework for Few-Shot Knowledge Base Question Answering

Few-Shot Learning with Self-supervised Classifier for Complex Knowledge Base Question Answering.

KBQA: Learning Question Answering over QA Corpora and Knowledge Bases

Interactive-KBQA: Multi-Turn Interactions for Knowledge Base Question Answering with Large Language Models

Coarse-to-Careful: Seeking Semantic-related Knowledge for Open-domain Commonsense Question Answering

In-Context Learning for Knowledge Base Question Answering for Unmanned Systems based on Large Language Models

ChatKBQA: A Generate-then-Retrieve Framework for Knowledge Base Question Answering with Fine-tuned Large Language Models

Make a Choice! Knowledge Base Question Answering with In-Context Learning

Bridging the Language Gap: Knowledge Injected Multilingual Question Answering

Chinese Knowledge Base Question Answering by Attention-Based Multi-Granularity Model

Cross-Lingual Question Answering over Knowledge Base as Reading Comprehension

Hybrid Question Answering over Knowledge Base and Free Text.

Program Transfer and Ontology Awareness for Semantic Parsing in KBQA

Topic enhanced deep structured semantic models for knowledge base question answering

Bridging the KB-Text Gap: Leveraging Structured Knowledge-aware Pre-training for KBQA

A Complex KBQA System using Multiple Reasoning Paths

How Question Generation Can Help Question Answering over Knowledge Base