A Principled Framework for Knowledge-enhanced Large Language Model

Saizhuo Wang,Zhihan Liu,Zhaoran Wang,Jian Guo
2023-11-19
Abstract:Large Language Models (LLMs) are versatile, yet they often falter in tasks requiring deep and reliable reasoning due to issues like hallucinations, limiting their applicability in critical scenarios. This paper introduces a rigorously designed framework for creating LLMs that effectively anchor knowledge and employ a closed-loop reasoning process, enhancing their capability for in-depth analysis. We dissect the framework to illustrate the contribution of each component to the LLMs' performance, offering a theoretical assurance of improved reasoning under well-defined assumptions.
Computation and Language,Artificial Intelligence
What problem does this paper attempt to address?
The paper primarily addresses the issues encountered by large language models (LLMs) in tasks requiring deep and reliable reasoning by proposing a principled framework. Specifically, the paper points out that although current large language models have achieved significant success in natural language understanding and generation, and are widely used to build language model agents (LLMs Agents) capable of handling complex tasks, they still have limitations when it comes to knowledge-intensive tasks. These tasks often require complex chains of reasoning and a high degree of factual accuracy. However, large language models perform poorly on these tasks due to their reliance on precise prompt engineering, limited context windows, and a tendency to generate unverified information (i.e., "hallucinations"). To address these issues, the paper proposes a carefully designed framework aimed at creating large language models that can effectively anchor knowledge and adopt a closed-loop reasoning process to enhance their ability to conduct in-depth analysis. The framework includes two levels of interactive processes: 1. **Inner Reasoning Loop**: This part focuses on the interaction between the user's natural language queries and external knowledge bases. By continuously retrieving information from the knowledge base, responses are formed until enough information is gathered to answer the user's question or a predefined stopping condition is met. 2. **Outer Feedback Loop**: This part involves the interaction between the LLM agent and the real-world environment, which includes user feedback to improve the agent's future responses. This allows the agent's capabilities to dynamically evolve and adapt to different domains. The paper also discusses in detail how different components of the framework affect the performance of large language models and provides theoretical guarantees that, under reasonable assumptions, the framework can improve reasoning capabilities. Additionally, the paper introduces several different types of LLM and knowledge base combinations, including knowledge graph-dominated approaches, LLM-dominated approaches, and hybrid approaches, and analyzes the characteristics of each method. In summary, this paper aims to enhance the performance of large language models in tasks requiring deep reasoning by proposing a comprehensive, principle-driven framework, particularly by integrating external knowledge sources and iterative learning mechanisms to reduce the occurrence of "hallucinations" and improve factual accuracy.