Abstract:Efficient knowledge management plays a pivotal role in augmenting both the operational efficiency and the innovative capacity of businesses and organizations. By indexing knowledge through vectorization, a variety of knowledge retrieval methods have emerged, significantly enhancing the efficacy of knowledge management systems. Recently, the rapid advancements in generative natural language processing technologies paved the way for generating precise and coherent answers after retrieving relevant documents tailored to user queries. However, for enterprise knowledge bases, assembling extensive training data from scratch for knowledge retrieval and generation is a formidable challenge due to the privacy and security policies of private data, frequently entailing substantial costs. To address the challenge above, in this paper, we propose EKRG, a novel Retrieval-Generation framework based on large language models (LLMs), expertly designed to enable question-answering for Enterprise Knowledge bases with limited annotation costs. Specifically, for the retrieval process, we first introduce an instruction-tuning method using an LLM to generate sufficient document-question pairs for training a knowledge retriever. This method, through carefully designed instructions, efficiently generates diverse questions for enterprise knowledge bases, encompassing both fact-oriented and solution-oriented knowledge. Additionally, we develop a relevance-aware teacher-student learning strategy to further enhance the efficiency of the training process. For the generation process, we propose a novel chain of thought (CoT) based fine-tuning method to empower the LLM-based generator to adeptly respond to user questions using retrieved documents. Finally, extensive experiments on real-world datasets have demonstrated the effectiveness of our proposed framework.

Logical Form Generation via Multi-task Learning for Complex Question Answering over Knowledge Bases.

Interactive-KBQA: Multi-Turn Interactions for Knowledge Base Question Answering with Large Language Models

How Question Generation Can Help Question Answering over Knowledge Base

ChatKBQA: A Generate-then-Retrieve Framework for Knowledge Base Question Answering with Fine-tuned Large Language Models

Complex Knowledge Base Question Answering: A Survey

How Proficient Are Large Language Models in Formal Languages? An In-Depth Insight for Knowledge Base Question Answering

In-Context Learning for Knowledge Base Question Answering for Unmanned Systems based on Large Language Models

KBQA: Learning Question Answering over QA Corpora and Knowledge Bases

Effective Search of Logical Forms for Weakly Supervised Knowledge-Based Question Answering

LFKQG: A Controlled Generation Framework with Local Fine-tuning for Question Generation over Knowledge Bases.

Complex Knowledge Base Question Answering for Intelligent Bridge Management Based on Multi-Task Learning and Cross-Task Constraints

A Complex KBQA System using Multiple Reasoning Paths

LB-KBQA: Large-language-model and BERT based Knowledge-Based Question and Answering System

Calculating Question Similarity is Enough: A New Method for KBQA Tasks

Structured Knowledge Grounding for Question Answering

Code-Style In-Context Learning for Knowledge-Based Question Answering

Generate-on-Graph: Treat LLM as both Agent and KG in Incomplete Knowledge Graph Question Answering

Retrieve-Rewrite-Answer: A KG-to-Text Enhanced LLMs Framework for Knowledge Graph Question Answering

EffiQA: Efficient Question-Answering with Strategic Multi-Model Collaboration on Knowledge Graphs

RnG-KBQA: Generation Augmented Iterative Ranking for Knowledge Base Question Answering

Enhancing Question Answering for Enterprise Knowledge Bases using Large Language Models