Abstract:The impressive performance of proprietary LLMs like GPT4 in code generation has led to a trend to replicate these capabilities in open-source models through knowledge distillation (e.g. Code Evol-Instruct). However, these efforts often neglect the crucial aspect of response quality, relying heavily on teacher models for direct response distillation. This paradigm, especially for complex instructions, can degrade the quality of synthesized data, compromising the knowledge distillation process. To this end, our study introduces the Adaptive Modular Response Evolution (AMR-Evol) framework, which employs a two-stage process to refine response distillation. The first stage, modular decomposition, breaks down the direct response into more manageable sub-modules. The second stage, adaptive response evolution, automatically evolves the response with the related function modules. Our experiments with three popular code benchmarks (HumanEval, MBPP, and EvalPlus) attest to the superiority of the AMR-Evol framework over baseline response distillation methods. By comparing with the open-source Code LLMs trained on a similar scale of data, we observed performance enhancements: more than +3.0 points on HumanEval-Plus and +1.0 points on MBPP-Plus, which underscores the effectiveness of our framework. Our codes are available at <a class="link-external link-https" href="https://github.com/ChiYeungLaw/AMR-Evol" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the insufficient response quality of existing knowledge distillation methods in code - generation tasks, especially when dealing with complex instructions. Specifically: 1. **Limitations of Direct - Response Distillation**: Existing methods (such as Code Evol - Instruct) rely on the teacher model to directly generate code responses. However, this method is prone to producing low - quality responses when dealing with complex instructions, thus affecting the learning effect of the student model. 2. **Importance of Response Quality**: High - quality code responses are crucial for effective knowledge distillation. However, current methods often overlook this point, leading to a decline in the quality of synthetic data and, in turn, affecting the effectiveness of knowledge distillation. 3. **Challenges of Complex Instructions**: As the complexity of instructions increases, the performance of direct distillation methods will decline significantly, especially in cases where precise logic and details are required. To solve these problems, the author introduced a new framework - **Adaptive Modular Response Evolution (AMR - Evol)**. This framework improves the quality of response distillation through two stages: - **Modular Decomposition (MD)**: Decompose complex code instructions into smaller and more manageable sub - modules, enabling the teacher model to solve these sub - modules step by step instead of generating a complete solution at once. - **Adaptive Response Evolution (ARE)**: Utilize an auxiliary functional module database to store and reuse verified modules. When generating responses, combine relevant modules in the existing module library to reduce dependence on the teacher model and improve the quality of responses. Through these two stages, the AMR - Evol framework aims to improve the response quality in code - generation tasks, thereby enhancing the knowledge distillation effect of the student model. Experimental results show that AMR - Evol significantly outperforms other methods in multiple benchmark tests, especially on complex instructions. In summary, the core problem of this paper is to improve the effectiveness of knowledge distillation by improving the response quality in code - generation tasks, especially for the processing of complex instructions.

AMR-Evol: Adaptive Modular Response Evolution Elicits Better Knowledge Distillation for Large Language Models in Code Generation

SelfEvolve: A Code Evolution Framework via Large Language Models

Personalised Distillation: Empowering Open-Sourced LLMs with Adaptive Learning for Code Generation

Towards more realistic evaluation of LLM-based code generation: an experimental study and beyond

Evolutionary Contrastive Distillation for Language Model Alignment

Evolving Knowledge Distillation with Large Language Models and Active Learning

Lion: Adversarial Distillation of Proprietary Large Language Models

Using Advanced LLMs to Enhance Smaller LLMs: An Interpretable Knowledge Distillation Approach

Instruction Fusion: Advancing Prompt Evolution through Hybridization

Ever Evolving Evaluator (EV3): Towards Flexible and Reliable Meta-Optimization for Knowledge Distillation

A Survey on Knowledge Distillation of Large Language Models

Revolve: Optimizing AI Systems by Tracking Response Evolution in Textual Optimization

CodeACT: Code Adaptive Compute-efficient Tuning Framework for Code LLMs

AlchemistCoder: Harmonizing and Eliciting Code Capability by Hindsight Tuning on Multi-source Data

DolphCoder: Echo-Locating Code Large Language Models with Diverse and Multi-Objective Instruction Tuning

Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM

Auto-Evolve: Enhancing Large Language Model's Performance via Self-Reasoning Framework

CourseGPT-zh: an Educational Large Language Model Based on Knowledge Distillation Incorporating Prompt Optimization

Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation

CodeLutra: Boosting LLM Code Generation via Preference-Guided Refinement

Knowledge Distillation Using Frontier Open-source LLMs: Generalizability and the Role of Synthetic Data