Abstract:Instruction-following is crucial for building AI agents with large language models (LLMs), as these models must adhere strictly to user-provided constraints and guidelines. However, LLMs often fail to follow even simple and clear instructions. To improve instruction-following behavior and prevent undesirable outputs, a deeper understanding of how LLMs' internal states relate to these outcomes is required. Our analysis of LLM internal states reveal a dimension in the input embedding space linked to successful instruction-following. We demonstrate that modifying representations along this dimension improves instruction-following success rates compared to random changes, without compromising response quality. Further investigation reveals that this dimension is more closely related to the phrasing of prompts rather than the inherent difficulty of the task or instructions. This discovery also suggests explanations for why LLMs sometimes fail to follow clear instructions and why prompt engineering is often effective, even when the content remains largely unchanged. This work provides insight into the internal workings of LLMs' instruction-following, paving the way for reliable LLM agents.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the difficulties that large language models (LLMs) have in following instructions. Although these models perform well on many tasks, they are often unable to follow simple and explicit instructions provided by users, which limits their effectiveness as personal AI agents. Specifically, the paper focuses on the following aspects: 1. **Understanding the relationship between the internal states of LLMs and instruction - following**: In order to improve the instruction - following behavior of LLMs and prevent unwanted outputs, it is necessary to deeply understand how the internal states of these models affect the results of their instruction - following. By analyzing the internal states of LLMs, the paper has discovered a dimension in the input embedding space that is related to successful instruction - following. 2. **Increasing the success rate of instruction - following**: The paper shows that by modifying the representation along this specific dimension, the success rate of instruction - following can be increased without sacrificing the response quality. This means that the ability of the model to follow instructions can be improved by adjusting its internal representation. 3. **Explaining why LLMs sometimes fail**: Further research indicates that this dimension is more related to the way the prompt is formulated rather than the inherent difficulty of the task or instruction. This explains why LLMs sometimes fail to follow clear instructions and why prompt engineering is still effective when the content does not change much. 4. **Promoting the development of reliable LLM agents**: By revealing the internal mechanisms of LLMs in instruction - following, this work provides a theoretical basis and practical methods for developing more reliable and trustworthy AI agents. In summary, the paper aims to improve the ability of LLMs to follow user instructions by analyzing and improving their internal representations, thereby promoting the development of safer and more trustworthy AI agents.

Do LLMs "know" internally when they follow instructions?

Do LLMs "know" internally when they follow instructions?

Do LLMs estimate uncertainty well in instruction-following?

LLM-driven Instruction Following: Progresses and Concerns

An Incomplete Loop: Instruction Inference, Instruction Following, and In-context Learning in Language Models

Find the Intention of Instruction: Comprehensive Evaluation of Instruction Understanding for Large Language Models

Nevermind: Instruction Override and Moderation in Large Language Models

Can Large Language Models Understand Real-World Complex Instructions?

Beyond Instruction Following: Evaluating Inferential Rule Following of Large Language Models

Thinking LLMs: General Instruction Following with Thought Generation

Evaluating Large Language Models at Evaluating Instruction Following

Can LLMs Separate Instructions From Data? And What Do We Even Mean By That?

LLM Self-Correction with DeCRIM: Decompose, Critique, and Refine for Enhanced Following of Instructions with Multiple Constraints

How You Prompt Matters! Even Task-Oriented Constraints in Instructions Affect LLM-Generated Text Detection

From Words to Actions: Unveiling the Theoretical Underpinnings of LLM-Driven Autonomous Systems

Can LLMs Follow Simple Rules?

Can LLMs plan paths with extra hints from solvers?

Instruction-Following Speech Recognition

Enhancing Large Language Models Against Inductive Instructions with Dual-critique Prompting

From Complex to Simple: Enhancing Multi-Constraint Complex Instruction Following Ability of Large Language Models