Abstract:In human-written articles, we often leverage the subtleties of text style, such as bold and italics, to guide the attention of readers. These textual emphases are vital for the readers to grasp the conveyed information. When interacting with large language models (LLMs), we have a similar need -- steering the model to pay closer attention to user-specified information, e.g., an instruction. Existing methods, however, are constrained to process plain text and do not support such a mechanism. This motivates us to introduce PASTA -- Post-hoc Attention STeering Approach, a method that allows LLMs to read text with user-specified emphasis marks. To this end, PASTA identifies a small subset of attention heads and applies precise attention reweighting on them, directing the model attention to user-specified parts. Like prompting, PASTA is applied at inference time and does not require changing any model parameters. Experiments demonstrate that PASTA can substantially enhance an LLM's ability to follow user instructions or integrate new knowledge from user inputs, leading to a significant performance improvement on a variety of tasks, e.g., an average accuracy improvement of 22% for LLAMA-7B. Our code is publicly available at <a class="link-external link-https" href="https://github.com/QingruZhang/PASTA" rel="external noopener nofollow">this https URL</a> .

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to guide the model to pay more attention to user - specified information, such as instructions, when interacting with large - language models (LLMs). Existing methods usually deal with plain text and do not support such an emphasis mechanism. This leads to the difficulty for LLMs to understand the user's intention when dealing with complex instructions, long - background contexts or knowledge conflicts in the context. To solve these problems, the authors propose PASTA (Post - hoc Attention STeering Approach), a post - processing attention - guiding method that allows users to guide the model to focus on specific parts of the content through emphasis markers. Specifically, PASTA achieves this goal in the following ways: 1. **Post - processing attention guidance**: PASTA precisely re - weights the attention of selected attention heads during the inference stage, enhancing the attention scores of user - specified parts while reducing the scores of other parts. This can guide the model to pay more attention to user - specified information, thereby better following user instructions or integrating new knowledge. 2. **Multi - task model configuration file**: To select effective attention heads, PASTA uses an efficient model configuration file algorithm. This algorithm evaluates small - sample training sets of multiple tasks to identify which attention heads can effectively improve multi - task performance when guided. These selected attention heads can also show good effects in unseen tasks. The experimental results show that PASTA can significantly improve the performance of LLMs on a variety of tasks, such as: - **JSON formatting**: Improve the model's ability to generate JSON - format output that meets user requirements. - **Pronoun conversion**: Improve the model's ability to replace pronouns according to user instructions. - **Knowledge conflicts in the context**: Improve the model's ability to generate consistent text when new facts emerge. - **Long - context understanding**: Improve the model's ability to capture key information in long - background contexts. In conclusion, through the post - processing attention - guiding mechanism, PASTA effectively solves the deficiencies of existing LLMs in dealing with complex instructions, long - background contexts and knowledge conflicts, and improves the model's understanding and generation abilities.

Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMs

Model Tells Itself Where to Attend: Faithfulness Meets Automatic Attention Steering

LLMSteer: Improving Long-Context LLM Inference by Steering Attention on Reused Contexts

Unveiling and Harnessing Hidden Attention Sinks: Enhancing Large Language Models without Training through Attention Calibration

Semantics-Adaptive Activation Intervention for LLMs via Dynamic Steering Vectors

Attention Instruction: Amplifying Attention in the Middle via Prompting

Pedagogical Alignment of Large Language Models

Personalized Steering of Large Language Models: Versatile Steering Vectors Through Bi-directional Preference Optimization

Selective Prompt Anchoring for Code Generation

CogSteer: Cognition-Inspired Selective Layer Intervention for Efficient Semantic Steering in Large Language Models

Steering Llama 2 via Contrastive Activation Addition

Fortify the Shortest Stave in Attention: Enhancing Context Awareness of Large Language Models for Effective Tool Use

Aligning Large Language Models via Fine-grained Supervision

LLMs-as-Instructors: Learning from Errors Toward Automating Model Improvement

LLM-augmented Preference Learning from Natural Language

Prompting and Fine-Tuning Open-Sourced Large Language Models for Stance Classification

Adaptive Activation Steering: A Tuning-Free LLM Truthfulness Improvement Method for Diverse Hallucinations Categories

SciLitLLM: How to Adapt LLMs for Scientific Literature Understanding

Guiding Large Language Models via Directional Stimulus Prompting

Few-shot Steerable Alignment: Adapting Rewards and LLM Policies with Neural Processes

Evaluating Large Language Models at Evaluating Instruction Following