InstructZero: Efficient Instruction Optimization for Black-Box Large Language Models

Lichang Chen,Jiuhai Chen,Tom Goldstein,Heng Huang,Tianyi Zhou

2023-08-09

Abstract:Large language models~(LLMs) are instruction followers, but it can be challenging to find the best instruction for different situations, especially for black-box LLMs on which backpropagation is forbidden. Instead of directly optimizing the discrete instruction, we optimize a low-dimensional soft prompt applied to an open-source LLM to generate the instruction for the black-box LLM. On each iteration of the proposed method, which we call InstructZero, a soft prompt is converted into an instruction using the open-source LLM, which is then submitted to the black-box LLM for zero-shot evaluation, and the performance is sent to Bayesian optimization to produce new soft prompts improving the zero-shot performance. We evaluate InstructZero on different combinations of open-source LLMs and APIs including Vicuna and ChatGPT. Our results show that InstructZero outperforms SOTA auto-instruction methods across a variety of downstream tasks. Our code and data are publicly available at <a class="link-external link-https" href="https://github.com/Lichang-Chen/InstructZero" rel="external noopener nofollow">this https URL</a>.

Artificial Intelligence

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to find the optimal instructions for black - box large language models (LLMs), especially in the zero - sample setting. Specifically, the paper focuses on how to optimize the instructions used to guide these models to perform specific tasks without allowing back - propagation for these black - box models. Since the internal mechanisms of black - box LLMs (such as ChatGPT and GPT - 4) are opaque, directly optimizing discrete instruction texts in a high - dimensional, discrete and highly - structured space is very difficult. Therefore, the paper proposes a new method named INSTRUCT ZERO. By optimizing a low - dimensional soft prompt, it uses open - source LLMs to generate human - readable and task - related instructions, and then submits these instructions to the black - box LLM for zero - sample evaluation, so as to improve the performance of the black - box LLM on various downstream tasks. The main contribution of the paper lies in providing an effective and efficient solution that can automatically search or optimize instructions for any given task, thereby significantly improving the performance of black - box LLMs on multiple tasks, especially those tasks that require precise instructions. Verified by experiments, INSTRUCT ZERO outperforms existing automatic instruction - optimization methods in multiple benchmark tests.

InstructZero: Efficient Instruction Optimization for Black-Box Large Language Models

Use Your INSTINCT: INSTruction optimization for LLMs usIng Neural bandits Coupled with Transformers

Auto-Instruct: Automatic Instruction Generation and Ranking for Black-Box Language Models

Evaluating the Zero-shot Robustness of Instruction-tuned Language Models

VisLingInstruct: Elevating Zero-Shot Learning in Multi-Modal Language Models with Autonomous Instruction Optimization

InverseCoder: Self-improving Instruction-Tuned Code LLMs with Inverse-Instruct

Instruction Distillation Makes Large Language Models Efficient Zero-shot Rankers

Unlocking Black-Box Prompt Tuning Efficiency Via Zeroth-Order Optimization

Evaluating Instruction-Tuned Large Language Models on Code Comprehension and Generation

MLAN: Language-Based Instruction Tuning Improves Zero-Shot Generalization of Multimodal Large Language Models

DPZero: Private Fine-Tuning of Language Models without Backpropagation

InstOptima: Evolutionary Multi-objective Instruction Optimization via Large Language Model-based Instruction Operators

BayLing: Bridging Cross-lingual Alignment and Instruction Following through Interactive Translation for Large Language Models

Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmark

LLMs-as-Instructors: Learning from Errors Toward Automating Model Improvement

From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuning

A Setwise Approach for Effective and Highly Efficient Zero-shot Ranking with Large Language Models

LPZero: Language Model Zero-cost Proxy Search from Zero

Instruction Mining: Instruction Data Selection for Tuning Large Language Models

Finetuned Language Models Are Zero-Shot Learners

Inference Performance Optimization for Large Language Models on CPUs