Abstract:Retrieval-augmented code generation utilizes Large Language Models as the generator and significantly expands their code generation capabilities by providing relevant code, documentation, and more via the retriever. The current approach suffers from two primary limitations: 1) information redundancy. The indiscriminate inclusion of redundant information can result in resource wastage and may misguide generators, affecting their effectiveness and efficiency. 2) preference gap. Due to different optimization objectives, the retriever strives to procure code with higher ground truth similarity, yet this effort does not substantially benefit the generator. The retriever and the generator may prefer different golden code, and this gap in preference results in a suboptimal design. Additionally, differences in parameterization knowledge acquired during pre-training result in varying preferences among different generators. To address these limitations, in this paper, we propose RRG (Retrieve, Refactor, Generate), a novel framework for effective and efficient code generation. This framework introduces a code refactorer module between the retriever and the generator to bridge them. The refactoring process transforms the raw retrieved code into a more concise, efficient, and model-friendly version. It eliminates redundant information and noise, reducing the input length. Consequently, the generator receives higher-quality context, enabling it to produce more accurate results with lower inference costs. We conducted comprehensive experiments on multiple datasets. In the experiments, we confirmed the existence of a preference gap between the retriever and the generator, and RRG effectively bridges this gap. Specifically, RRG achieved significant performance improvements, with increases of up to 28% on EM, 13% on BLEU, and 6.8% on CodeBLEU.

Syntax-Aware Retrieval Augmented Code Generation

Generation-Augmented Query Expansion For Code Retrieval

PinNet: Pinpoint Instructive Information for Retrieval Augmented Code-to-Text Generation

Knowledge Distillation-Based Multilingual Code Retrieval

CodeGRAG: Bridging the Gap between Natural Language and Programming Language via Graphical Retrieval Augmented Generation

A Simple Retrieval-based Method for Code Comment Generation

Code Generation with Hybrid of Structural and Semantic Features Retrieval

Preference-Guided Refactored Tuning for Retrieval Augmented Code Generation

AugmentedCode: Examining the Effects of Natural Language Resources in Code Retrieval Models

CodeRAG-Bench: Can Retrieval Augment Code Generation?

Incorporating Domain Knowledge through Task Augmentation for Front-End JavaScript Code Generation

Tram: A Token-level Retrieval-augmented Mechanism for Source Code Summarization

Exploring Representation-Level Augmentation for Code Search

GenCode: A Generic Data Augmentation Framework for Boosting Deep Learning-Based Code Understanding

RRGcode: Deep hierarchical search-based code generation

Prompt-based Code Completion via Multi-Retrieval Augmented Generation

Code Search based on Context-aware Code Translation

FT2Ra: A Fine-Tuning-Inspired Approach to Retrieval-Augmented Code Completion

Retrieval-Augmented Generation for Natural Language Processing: A Survey

Code2Text: Dual Attention Syntax Annotation Networks for Structure-Aware Code Translation