Abstract:With the rise of powerful closed-sourced LLMs (ChatGPT, GPT-4), there are increasing interests in distilling the capabilies of close-sourced LLMs to smaller open-sourced LLMs. Previous distillation methods usually prompt ChatGPT to generate a set of instructions and answers, for the student model to learn. However, such standard distillation approach neglects the merits and conditions of the student model. Inspired by modern teaching principles, we design a personalised distillation process, in which the student attempts to solve a task first, then the teacher provides an adaptive refinement for the student to improve. Instead of feeding the student with teacher's prior, personalised distillation enables personalised learning for the student model, as it only learns on examples it makes mistakes upon and learns to improve its own solution. On code generation, personalised distillation consistently outperforms standard distillation with only one third of the data. With only 2.5-3K personalised examples that incur a data-collection cost of 4-6$, we boost CodeGen-mono-16B by 7% to achieve 36.4% pass@1 and StarCoder by 12.2% to achieve 45.8% pass@1 on HumanEval.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: how to effectively transfer the capabilities of powerful closed - source large - language models (such as ChatGPT, GPT - 4) to smaller open - source large - language models, especially in code - generation tasks. Traditional knowledge - distillation methods usually let the closed - source model generate a series of instructions and answers for the student model to learn. However, this method ignores the unique capabilities and conditions of the student model, resulting in low learning efficiency. To solve this problem, the paper proposes a personalised distillation method. This method allows the student model to first attempt to solve the problem, and then the teacher model provides a personalised improvement plan based on the student's attempt. This interactive and personalised learning method aims to improve the learning efficiency and effectiveness of the student model. Specifically, the main contributions of the paper include: 1. **Personalised distillation framework**: A three - step personalised distillation process is designed, including the student model attempting to solve the problem, performing unit tests to obtain feedback, and the teacher model providing an improvement plan based on the feedback. 2. **Dataset construction**: Through interaction with ChatGPT, a standard distillation dataset and a personalised distillation dataset are constructed. 3. **Experimental verification**: Extensive experiments were carried out on two datasets, HumanEval and MBPP, to verify the effectiveness of the personalised distillation method. The results show that the personalised distillation method can still significantly improve the performance of the student model when the amount of data is only one - third of that of the standard distillation method. Through these contributions, the paper not only provides a new knowledge - distillation method, but also provides new ideas on how to use the capabilities of closed - source large - language models more efficiently in practical applications.

Personalised Distillation: Empowering Open-Sourced LLMs with Adaptive Learning for Code Generation

Using Advanced LLMs to Enhance Smaller LLMs: An Interpretable Knowledge Distillation Approach

AMR-Evol: Adaptive Modular Response Evolution Elicits Better Knowledge Distillation for Large Language Models in Code Generation

Lion: Adversarial Distillation of Proprietary Large Language Models

Knowledge Distillation Using Frontier Open-source LLMs: Generalizability and the Role of Synthetic Data

Beyond Answers: Transferring Reasoning Capabilities to Smaller LLMs Using Multi-Teacher Knowledge Distillation

Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes

Visual Editing with LLM-based Tool Chaining: An Efficient Distillation Approach for Real-Time Applications

Unlock the Power: Competitive Distillation for Multi-Modal Large Language Models

Teach LLMs to Personalize -- An Approach inspired by Writing Education

Adversarial Distillation for Learning with Privileged Provisions

CodeLutra: Boosting LLM Code Generation via Preference-Guided Refinement

Differentially Private Knowledge Distillation via Synthetic Text Generation

Effective Distillation of Table-based Reasoning Ability from LLMs

PLaD: Preference-based Large Language Model Distillation with Pseudo-Preference Pairs

Knowledge Distillation of LLM for Automatic Scoring of Science Education Assessments

GOLD: Generalized Knowledge Distillation via Out-of-Distribution-Guided Language Data Generation

MiniLLM: Knowledge Distillation of Large Language Models

Better than Your Teacher: LLM Agents that learn from Privileged AI Feedback

Distilling Instruction-following Abilities of Large Language Models with Task-aware Curriculum Planning

Mixed Distillation Helps Smaller Language Models Reason Better