Abstract:We introduce Directional Stimulus Prompting, a novel framework for guiding black-box large language models (LLMs) toward specific desired outputs. Instead of directly adjusting LLMs, our method employs a small tunable policy model (e.g., T5) to generate an auxiliary directional stimulus prompt for each input instance. These directional stimulus prompts act as nuanced, instance-specific hints and clues to guide LLMs in generating desired outcomes, such as including specific keywords in the generated summary. Our approach sidesteps the challenges of direct LLM tuning by optimizing the policy model to explore directional stimulus prompts that align LLMs with desired behaviors. The policy model can be optimized through 1) supervised fine-tuning using labeled data and 2) reinforcement learning from offline or online rewards based on the LLM's output. We assess our method across summarization, dialogue response generation, and chain-of-thought reasoning tasks. Our experiments demonstrate that the framework consistently improves LLMs' (e.g., ChatGPT, Codex, InstructGPT) performance on these supervised tasks using minimal labeled data. Notably, using just 80 dialogues on the MultiWOZ dataset, our approach enhances ChatGPT's performance by an impressive 41.4%, matching or surpassing some fully supervised start-of-the-art models. Additionally, the instance-specific chain-of-thought prompt generated by our approach improves InstructGPT's reasoning accuracy compared to human-crafted or automatically generated prompts. The code and data are publicly available at \url{<a class="link-external link-https" href="https://github.com/Leezekun/Directional-Stimulus-Prompting" rel="external noopener nofollow">this https URL</a>}.

Few-Shot Natural Language Inference Generation with PDD: Prompt and Dynamic Demonstration

Prompting-based Synthetic Data Generation for Few-Shot Question Answering

Effective Demonstration Annotation for In-Context Learning via Language Model-Based Determinantal Point Process

Adapting Prompt for Few-shot Table-to-Text Generation

Prompt Highlighter: Interactive Control for Multi-Modal LLMs

Demonstration Augmentation for Zero-shot In-context Learning

DPIC: Decoupling Prompt and Intrinsic Characteristics for LLM Generated Text Detection

Towards Informative Few-Shot Prompt with Maximum Information Gain for In-Context Learning

Fairness-guided Few-shot Prompting for Large Language Models

Few-Shot Data Synthesis for Open Domain Multi-Hop Question Answering

Mixture of Soft Prompts for Controllable Data Generation

Prompt-based Logical Semantics Enhancement for Implicit Discourse Relation Recognition

Synthetic Prompting: Generating Chain-of-Thought Demonstrations for Large Language Models

Unified Prompt Learning Makes Pre-Trained Language Models Better Few-Shot Learners

Few-Shot Table-to-Text Generation with Prompt Planning and Knowledge Memorization

Guiding Large Language Models via Directional Stimulus Prompting

Enhancing Few-shot Text-to-SQL Capabilities of Large Language Models: A Study on Prompt Design Strategies

Prompt2Model: Generating Deployable Models from Natural Language Instructions

Set-Based Prompting: Provably Solving the Language Model Order Dependency Problem

Few-shot Natural Language Generation for Task-Oriented Dialog

Controllable Generation of Dialogue Acts for Dialogue Systems via Few-Shot Response Generation and Ranking