Abstract:Most large language models (LLMs), especially chat models, do not undergo updates post-deployment, which results in a lack of awareness of constantly changing new real-time information. External knowledge retrieval methods exist to enhance these models, but for high-traffic chat models, the extensive use of search engine API resources can be costly. In our work, we study the performance of LLMs in answering tasks involving real-time factual knowledge. Due to the lack of datasets for Q&A involving the latest real-time information, we first carefully constructed NewlyQA, a new dynamic QA benchmark including questions about world knowledge whose answers evolve over time. Inspired by human learning methods, we propose FastLearn, a simple rapid learning method that enhances model responses by merging relevant, up-to-date information retrieved from search engine APIs (filtered through our Time-Prompt mechanism) into prompts, followed by Lora fine-tuning to “inject” new knowledge into LLMs via external parameters, significantly improving performance on FreshQA, NewlyQA and OpenDataEval. Our experiments show that FastLearn outperforms most other search engine augmented prompt methods, like FreshPrompt, Self-Ask, and the commercial PERPLEXITY.AI. Additionally, we further analyze the two main mechanisms of FastLearn, Time-Prompt and Self-Correction, finding that both play key roles in enhancing model answer performance. To facilitate future research, we will open-source NewlyQA and commit to regularly updating its answers.

Reciprocal Learning of Knowledge Retriever and Response Ranker for Knowledge-Grounded Conversations.

Learning Multi-turn Response Selection in Grounded Dialogues with Reinforced Knowledge and Context Distillation

PK-ICR: Persona-Knowledge Interactive Multi-Context Retrieval for Grounded Dialogue

Training Two-Stage Knowledge-Grounded Dialogues with Attention Feedback

CORE: Cooperative Training of Retriever-Reranker for Effective Dialogue Response Selection

Learning to Detect Relevant Contexts and Knowledge for Response Selection in Retrieval-based Dialogue Systems

Knowledge-Grounded Dialogue with Reward-Driven Knowledge Selection

Dual-Feedback Knowledge Retrieval for Task-Oriented Dialogue Systems

Knowledge-Grounded Dialogue Generation with Pre-trained Language Models

Building an Efficient and Effective Retrieval-based Dialogue System via Mutual Learning

Improving Dialogue Generation via Proactively Querying Grounded Knowledge

FastLearn: A Rapid Learning Agent for Chat Models to Acquire Latest Knowledge

A Multi Domain Knowledge Enhanced Matching Network for Response Selection in Retrieval-Based Dialogue Systems

Multi-Grained Knowledge Retrieval for End-to-End Task-Oriented Dialog

Filtering before Iteratively Referring for Knowledge-Grounded Response Selection in Retrieval-Based Chatbots

Exploring Dense Retrieval for Dialogue Response Selection

Dynamically Retrieving Knowledge via Query Generation for Informative Dialogue Generation

Response Ranking with Deep Matching Networks and External Knowledge in Information-seeking Conversation Systems

Retrieval-Generation Alignment for End-to-End Task-Oriented Dialogue System

Coarse-to-Fine Knowledge Selection for Document Grounded Dialogs.

Improving Knowledge-Aware Dialogue Response Generation by Using Human-Written Prototype Dialogues.