Abstract:Understanding alignment techniques begins with comprehending zero-shot generalization brought by instruction tuning, but little of the mechanism has been understood. Existing work has largely been confined to the task level, without considering that tasks are artificially defined and, to LLMs, merely consist of tokens and representations. This line of research has been limited to examining transfer between tasks from a task-pair perspective, with few studies focusing on understanding zero-shot generalization from the perspective of the data itself. To bridge this gap, we first demonstrate through multiple metrics that zero-shot generalization during instruction tuning happens very early. Next, we investigate the facilitation of zero-shot generalization from both data similarity and granularity perspectives, confirming that encountering highly similar and fine-grained training data earlier during instruction tuning, without the constraints of defined "tasks", enables better generalization. Finally, we propose a more grounded training data arrangement method, Test-centric Multi-turn Arrangement, and show its effectiveness in promoting continual learning and further loss reduction. For the first time, we show that zero-shot generalization during instruction tuning is a form of similarity-based generalization between training and test data at the instance level. We hope our analysis will advance the understanding of zero-shot generalization during instruction tuning and contribute to the development of more aligned LLMs. Our code is released at <a class="link-external link-https" href="https://github.com/HBX-hbx/dynamics_of_zero-shot_generalization" rel="external noopener nofollow">this https URL</a>.

Enabling Natural Zero-Shot Prompting on Encoder Models via Statement-Tuning

Masked Siamese Prompt Tuning for Few-Shot Natural Language Understanding

Finetuned Language Models Are Zero-Shot Learners

MLAN: Language-Based Instruction Tuning Improves Zero-Shot Generalization of Multimodal Large Language Models

Enhancing Zero-Shot Crypto Sentiment With Fine-Tuned Language Model and Prompt Engineering

Go-tuning: Improving Zero-shot Learning Abilities of Smaller Language Models

What Makes Pre-trained Language Models Better Zero/Few-shot Learners?

M^2PT: Multimodal Prompt Tuning for Zero-shot Instruction Learning

M$^2$PT: Multimodal Prompt Tuning for Zero-shot Instruction Learning

A Zero-shot and Few-shot Study of Instruction-Finetuned Large Language Models Applied to Clinical and Biomedical Tasks

A Minimalist Prompt for Zero-Shot Policy Learning

InstructZero: Efficient Instruction Optimization for Black-Box Large Language Models

Leveraging Zero-Shot Prompting for Efficient Language Model Distillation

Scaled Prompt-Tuning for Few-Shot Natural Language Generation

Zero-Shot Generalization during Instruction Tuning: Insights from Similarity and Granularity

Meta-Prompting for Automating Zero-shot Visual Recognition with LLMs

Instances Need More Care: Rewriting Prompts for Instances with LLMs in the Loop Yields Better Zero-Shot Performance

Prompting Encoder Models for Zero-Shot Classification: A Cross-Domain Study in Italian

Zero-to-Strong Generalization: Eliciting Strong Capabilities of Large Language Models Iteratively without Gold Labels

UPRISE: Universal Prompt Retrieval for Improving Zero-Shot Evaluation

Evaluating the Zero-shot Robustness of Instruction-tuned Language Models