Abstract:Knowledge base question answering (KBQA) is a critical yet challenging task due to the vast number of entities within knowledge bases and the diversity of natural language questions posed by users. Unfortunately, the performance of most KBQA models tends to decline significantly in real-world scenarios where high-quality annotated data is insufficient. To mitigate the burden associated with manual annotation, we introduce FlexKBQA by utilizing Large Language Models (LLMs) as program translators for addressing the challenges inherent in the few-shot KBQA task. Specifically, FlexKBQA leverages automated algorithms to sample diverse programs, such as SPARQL queries, from the knowledge base, which are subsequently converted into natural language questions via LLMs. This synthetic dataset facilitates training a specialized lightweight model for the KB. Additionally, to reduce the barriers of distribution shift between synthetic data and real user questions, FlexKBQA introduces an executionguided self-training method to iterative leverage unlabeled user questions. Furthermore, we explore harnessing the inherent reasoning capability of LLMs to enhance the entire framework. Consequently, FlexKBQA delivers substantial flexibility, encompassing data annotation, deployment, and being domain agnostic. Through extensive experiments on GrailQA, WebQSP, and KQA Pro, we observe that under the few-shot even the more challenging zero-shot scenarios, FlexKBQA achieves impressive results with a few annotations, surpassing all previous baselines and even approaching the performance of supervised models, achieving a remarkable 93% performance relative to the fully-supervised models. We posit that FlexKBQA represents a significant advancement towards exploring better integration of large and lightweight models. The code is open-sourced.

FC-KBQA: A Fine-to-Coarse Composition Framework for Knowledge Base Question Answering

Coarse-to-Careful: Seeking Semantic-related Knowledge for Open-domain Commonsense Question Answering

LFKQG: A Controlled Generation Framework with Local Fine-tuning for Question Generation over Knowledge Bases.

ChatKBQA: A Generate-then-Retrieve Framework for Knowledge Base Question Answering with Fine-tuned Large Language Models

Interactive-KBQA: Multi-Turn Interactions for Knowledge Base Question Answering with Large Language Models

EffiQA: Efficient Question-Answering with Strategic Multi-Model Collaboration on Knowledge Graphs

FiTs: Fine-grained Two-stage Training for Knowledge-aware Question Answering.

FlexKBQA: A Flexible LLM-Powered Framework for Few-Shot Knowledge Base Question Answering

A Survey on Complex Knowledge Base Question Answering: Methods, Challenges and Solutions

Fusing Essential Knowledge for Text-Based Open-Domain Question Answering

Structured Knowledge Grounding for Question Answering

KBQA: Learning Question Answering over QA Corpora and Knowledge Bases

FeQA: Fusion and enhancement of multi-source knowledge on question answering

Hybrid Question Answering over Knowledge Base and Free Text.

Chinese Knowledge Base Question Answering by Attention-Based Multi-Granularity Model

A State-transition Framework to Answer Complex Questions over Knowledge Base

An In-Context Schema Understanding Method for Knowledge Base Question Answering

Make a Choice! Knowledge Base Question Answering with In-Context Learning

Question Decomposition Tree for Answering Complex Questions over Knowledge Bases

How Proficient Are Large Language Models in Formal Languages? An In-Depth Insight for Knowledge Base Question Answering

CoQ:AN Empirical Framework for Multi-hop Question Answering Empowered by Large Language Models