Abstract:Misunderstandings arise not only in interpersonal communication but also between humans and Large Language Models (LLMs). Such discrepancies can make LLMs interpret seemingly unambiguous questions in unexpected ways, yielding incorrect responses. While it is widely acknowledged that the quality of a prompt, such as a question, significantly impacts the quality of the response provided by LLMs, a systematic method for crafting questions that LLMs can better comprehend is still underdeveloped. In this paper, we present a method named `Rephrase and Respond' (RaR), which allows LLMs to rephrase and expand questions posed by humans and provide responses in a single prompt. This approach serves as a simple yet effective prompting method for improving performance. We also introduce a two-step variant of RaR, where a rephrasing LLM first rephrases the question and then passes the original and rephrased questions together to a different responding LLM. This facilitates the effective utilization of rephrased questions generated by one LLM with another. Our experiments demonstrate that our methods significantly improve the performance of different models across a wide range to tasks. We further provide a comprehensive comparison between RaR and the popular Chain-of-Thought (CoT) methods, both theoretically and empirically. We show that RaR is complementary to CoT and can be combined with CoT to achieve even better performance. Our work not only contributes to enhancing LLM performance efficiently and effectively but also sheds light on a fair evaluation of LLM capabilities. Data and codes are available at <a class="link-external link-https" href="https://github.com/uclaml/Rephrase-and-Respond" rel="external noopener nofollow">this https URL</a>.

Refusal in LLMs is an Affine Function

AdaRefiner: Refining Decisions of Language Models with Adaptive Feedback

Programming Refusal with Conditional Activation Steering

Applying Refusal-Vector Ablation to Llama 3.1 70B Agents

Refusal in Language Models Is Mediated by a Single Direction

Rethinking harmless refusals when fine-tuning foundation models

Learn to Refuse: Making Large Language Models More Controllable and Reliable through Knowledge Scope Limitation and Refusal Mechanism

Cannot or Should Not? Automatic Analysis of Refusal Composition in IFT/RLHF Datasets and Refusal Behavior of Black-Box LLMs

Robust LLM safeguarding via refusal feature adversarial training

Refuse Whenever You Feel Unsafe: Improving Safety in LLMs via Decoupled Refusal Training

Enhancing Adversarial Resistance in LLMs with Recursion

Refusing Safe Prompts for Multi-modal Large Language Models

Editing Arbitrary Propositions in LLMs without Subject Labels

Self and Cross-Model Distillation for LLMs: Effective Methods for Refusal Pattern Alignment

Can LLMs Follow Simple Rules?

Understanding and Mitigating Language Confusion in LLMs

Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselves

DROJ: A Prompt-Driven Attack against Large Language Models

Does Refusal Training in LLMs Generalize to the Past Tense?

Steering Language Model Refusal with Sparse Autoencoders

Steering Llama 2 via Contrastive Activation Addition