Abstract:Efficient knowledge management plays a pivotal role in augmenting both the operational efficiency and the innovative capacity of businesses and organizations. By indexing knowledge through vectorization, a variety of knowledge retrieval methods have emerged, significantly enhancing the efficacy of knowledge management systems. Recently, the rapid advancements in generative natural language processing technologies paved the way for generating precise and coherent answers after retrieving relevant documents tailored to user queries. However, for enterprise knowledge bases, assembling extensive training data from scratch for knowledge retrieval and generation is a formidable challenge due to the privacy and security policies of private data, frequently entailing substantial costs. To address the challenge above, in this paper, we propose EKRG, a novel Retrieval-Generation framework based on large language models (LLMs), expertly designed to enable question-answering for Enterprise Knowledge bases with limited annotation costs. Specifically, for the retrieval process, we first introduce an instruction-tuning method using an LLM to generate sufficient document-question pairs for training a knowledge retriever. This method, through carefully designed instructions, efficiently generates diverse questions for enterprise knowledge bases, encompassing both fact-oriented and solution-oriented knowledge. Additionally, we develop a relevance-aware teacher-student learning strategy to further enhance the efficiency of the training process. For the generation process, we propose a novel chain of thought (CoT) based fine-tuning method to empower the LLM-based generator to adeptly respond to user questions using retrieved documents. Finally, extensive experiments on real-world datasets have demonstrated the effectiveness of our proposed framework.

LFKQG: A Controlled Generation Framework with Local Fine-tuning for Question Generation over Knowledge Bases.

ChatKBQA: A Generate-then-Retrieve Framework for Knowledge Base Question Answering with Fine-tuned Large Language Models

How Question Generation Can Help Question Answering over Knowledge Base

CQG: A Simple and Effective Controlled Generation Framework for Multi-hop Question Generation

In-Context Learning for Knowledge Base Question Answering for Unmanned Systems based on Large Language Models

FC-KBQA: A Fine-to-Coarse Composition Framework for Knowledge Base Question Answering

RnG-KBQA: Generation Augmented Iterative Ranking for Knowledge Base Question Answering

Generate-on-Graph: Treat LLM as both Agent and KG in Incomplete Knowledge Graph Question Answering

FlexKBQA: A Flexible LLM-Powered Framework for Few-Shot Knowledge Base Question Answering

Formal Query Generation for Question Answering over Knowledge Bases

Knowledge-Enhanced Iterative Instruction Generation and Reasoning for Knowledge Base Question Answering

Logical Form Generation via Multi-task Learning for Complex Question Answering over Knowledge Bases.

Question-guided Knowledge Graph Re-scoring and Injection for Knowledge Graph Question Answering

A Neural Question Generation System Based on Knowledge Base.

Diversifying Question Generation over Knowledge Base via External Natural Questions

Simple Question Answering over Knowledge Graph Enhanced by Question Pattern Classification

Enhancing Question Answering for Enterprise Knowledge Bases using Large Language Models

UniKGQA: Unified Retrieval and Reasoning for Solving Multi-hop Question Answering Over Knowledge Graph

Interactive-KBQA: Multi-Turn Interactions for Knowledge Base Question Answering with Large Language Models

Few-shot In-context Learning for Knowledge Base Question Answering

Learning to Ask Appropriate Questions in Conversational Recommendation