Abstract:Large language models (LLMs) have achieved impressive success in text-formatted learning problems, and most popular LLMs have been deployed in a black-box fashion. Meanwhile, fine-tuning is usually necessary for a specific downstream task to obtain better performance, and this functionality is provided by the owners of the black-box LLMs. To fine-tune a black-box LLM, labeled data are always required to adjust the model parameters. However, in many real-world applications, LLMs can label textual datasets with even better quality than skilled human annotators, motivating us to explore the possibility of fine-tuning black-box LLMs with unlabeled data. In this paper, we propose unsupervised prompt learning for classification with black-box LLMs, where the learning parameters are the prompt itself and the pseudo labels of unlabeled data. Specifically, the prompt is modeled as a sequence of discrete tokens, and every token has its own to-be-learned categorical distribution. On the other hand, for learning the pseudo labels, we are the first to consider the in-context learning (ICL) capabilities of LLMs: we first identify reliable pseudo-labeled data using the LLM, and then assign pseudo labels to other unlabeled data based on the prompt, allowing the pseudo-labeled data to serve as in-context demonstrations alongside the prompt. Those in-context demonstrations matter: previously, they are involved when the prompt is used for prediction while they are not involved when the prompt is trained; thus, taking them into account during training makes the prompt-learning and prompt-using stages more consistent. Experiments on benchmark datasets show the effectiveness of our proposed algorithm. After unsupervised prompt learning, we can use the pseudo-labeled dataset for further fine-tuning by the owners of the black-box LLMs.

Black-box Prompt Tuning with Subspace Learning

Make Prompt-based Black-Box Tuning Colorful: Boosting Model Generalization from Three Orthogonal Perspectives

Black-Box Tuning for Language-Model-as-a-Service

Survival of the Most Influential Prompts: Efficient Black-Box Prompt Search via Clustering and Pruning

On Unsupervised Prompt Learning for Classification with Black-box Language Models

Learning from Contrastive Prompts: Automated Optimization and Adaptation

Making Pre-trained Language Models End-to-end Few-shot Learners with Contrastive Prompt Tuning

BayesPrompt: Prompting Large-Scale Pre-Trained Language Models on Few-shot Inference via Debiased Domain Abstraction

Exploring Low-dimensional Intrinsic Task Subspace Via Prompt Tuning.

Reliable Gradient-free and Likelihood-free Prompt Tuning

Dynamic Prompting: A Unified Framework for Prompt Tuning

Black-Box Prompt Optimization: Aligning Large Language Models without Model Training

Patch-Prompt Aligned Bayesian Prompt Tuning for Vision-Language Models

Q-Tuning: Queue-based Prompt Tuning for Lifelong Few-shot Language Learning

Hard Prompts Made Interpretable: Sparse Entropy Regularization for Prompt Tuning with RL

Automatic Prompt Optimization with "Gradient Descent" and Beam Search

An Exploration of Prompt Tuning on Generative Spoken Language Model for Speech Processing Tasks

Exploring Universal Intrinsic Task Subspace for Few-shot Learning Via Prompt Tuning

Exploring Universal Intrinsic Task Subspace Via Prompt Tuning

RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning