Abstract:Large language models (LLMs) often struggle to provide up-to-date information due to their one-time training and the constantly evolving nature of the world. To keep LLMs current, existing approaches typically involve continued pre-training on new documents. However, they frequently face difficulties in extracting stored knowledge. Motivated by the remarkable success of the Feynman Technique in efficient human learning, we introduce Self-Tuning, a learning framework aimed at improving an LLM's ability to effectively acquire new knowledge from raw documents through self-teaching. Specifically, we develop a Self-Teaching strategy that augments the documents with a set of knowledge-intensive tasks created in a self-supervised manner, focusing on three crucial aspects: memorization, comprehension, and self-reflection. In addition, we introduce three Wiki-Newpages-2023-QA datasets to facilitate an in-depth analysis of an LLM's knowledge acquisition ability concerning memorization, extraction, and reasoning. Extensive experimental results on Llama2 family models reveal that Self-Tuning consistently exhibits superior performance across all knowledge acquisition tasks and excels in preserving previous knowledge.

What problem does this paper attempt to address?

### The Problem the Paper Attempts to Solve The paper aims to address the issue of large language models (LLMs) struggling to acquire the latest knowledge. Since LLMs are typically trained once and the world is constantly changing, the knowledge stored in these models can quickly become outdated. Existing methods usually maintain the model's timeliness by continuing pre-training on new documents, but these methods often face difficulties in extracting and storing knowledge. Specifically, the paper proposes a learning framework called SELF-TUNING, which aims to improve LLMs' ability to effectively acquire new knowledge from raw documents through a self-teaching strategy. The SELF-TUNING framework includes the following three stages: 1. **Stage 1: Learning to Effectively Absorb Knowledge from Raw Documents** By combining a training document dataset (DDoc_train) and a related training question-answer dataset (DQA_train), along with a series of self-supervised knowledge-intensive tasks (DSelf_train), the model is trained to enhance its knowledge absorption capability. 2. **Stage 2: Learning New Knowledge While Reviewing Question-Answer Skills** The model is trained on an unseen test document dataset (DDoc_test), while also including the training question-answer dataset (DQA_train) to review and improve its question-answering ability, thereby enhancing its ability to extract knowledge from the test documents. 3. **Stage 3: Continuous Learning** By conducting subsequent training on the test document dataset (DDoc_test), the model ensures thorough absorption of new knowledge. Additionally, the paper introduces three Wiki-Newpages-2023-QA datasets to deeply analyze LLMs' knowledge acquisition capabilities in terms of memory, extraction, and reasoning. These datasets cover single-domain, multi-domain, and cross-domain settings, ensuring minimal overlap with the LLMs' pre-training corpus. Through these methods, the paper hopes to significantly improve LLMs' performance in multiple knowledge acquisition tasks while maintaining the accuracy of previously learned knowledge.

Self-Tuning: Instructing LLMs to Effectively Acquire New Knowledge through Self-Teaching

LLMs-as-Instructors: Learning from Errors Toward Automating Model Improvement

SELF-GUIDE: Better Task-Specific Instruction Following via Self-Synthetic Finetuning

CITING: Large Language Models Create Curriculum for Instruction Tuning

Selective Reflection-Tuning: Student-Selected Data Recycling for LLM Instruction-Tuning

LLMs Could Autonomously Learn Without External Supervision

Instruction-tuned Language Models are Better Knowledge Learners

Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizing

SELF: Self-Evolution with Language Feedback

TasTe: Teaching Large Language Models to Translate through Self-Reflection

KnowTuning: Knowledge-aware Fine-tuning for Large Language Models

Recursive Introspection: Teaching Language Model Agents How to Self-Improve

Self-training Large Language Models through Knowledge Detection

Language Model Self-improvement by Reinforcement Learning Contemplation

Reflection-Tuning: Data Recycling Improves LLM Instruction-Tuning

Teaching Language Models to Self-Improve by Learning from Language Feedback

SaySelf: Teaching LLMs to Express Confidence with Self-Reflective Rationales

Teaching Language Models to Self-Improve through Interactive Demonstrations

Can LLMs Learn by Teaching for Better Reasoning? A Preliminary Study

WisdomBot: Tuning Large Language Models with Artificial Intelligence Knowledge

Supervised Knowledge Makes Large Language Models Better In-context Learners