Abstract:Pretrained language models have shown strong effectiveness in code-related tasks, such as code retrieval, code generation, code summarization, and code completion tasks. In this paper, we propose COde assistaNt viA retrieval-augmeNted language model (CONAN), which aims to build a code assistant by mimicking the knowledge-seeking behaviors of humans during coding. Specifically, it consists of a code structure aware retriever (CONAN-R) and a dual-view code representation-based retrieval-augmented generation model (CONAN-G). CONAN-R pretrains CodeT5 using Code-Documentation Alignment and Masked Entity Prediction tasks to make language models code structure-aware and learn effective representations for code snippets and documentation. Then CONAN-G designs a dual-view code representation mechanism for implementing a retrieval-augmented code generation model. CONAN-G regards the code documentation descriptions as prompts, which help language models better understand the code semantics. Our experiments show that CONAN achieves convincing performance on different code generation tasks and significantly outperforms previous retrieval augmented code generation models. Our further analyses show that CONAN learns tailored representations for both code snippets and documentation by aligning code-documentation data pairs and capturing structural semantics by masking and predicting entities in the code data. Additionally, the retrieved code snippets and documentation provide necessary information from both program language and natural language to assist the code generation process. CONAN can also be used as an assistant for Large Language Models (LLMs), providing LLMs with external knowledge in shorter code document lengths to improve their effectiveness on various code tasks. It shows the ability of CONAN to extract necessary information and help filter out the noise from retrieved code documents.

Code Generation Using Self-Interactive Assistant

SelfEvolve: A Code Evolution Framework via Large Language Models

Self-collaboration Code Generation via ChatGPT

AgentCoder: Multi-Agent-based Code Generation with Iterative Testing and Optimisation

A Self-Iteration Code Generation Method Based on Large Language Models

Self-Programming Artificial Intelligence Using Code-Generating Language Models

AICoderEval: Improving AI Domain Code Generation of Large Language Models

AceCoder : An Effective Prompting Technique Specialized in Code Generation

AI-assisted coding: Experiments with GPT-4

StepCoder: Improve Code Generation with Reinforcement Learning from Compiler Feedback

StepCoder: Improving Code Generation with Reinforcement Learning from Compiler Feedback

Self-Edit: Fault-Aware Code Editor for Code Generation

ProgAI: Enhancing Code Generation with LLMs For Real World Challenges

OpenAi's GPT4 as coding assistant

Personality-Guided Code Generation Using Large Language Models

Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generation

Self-planning Code Generation with Large Language Models

Towards Enhancing In-Context Learning for Code Generation

How Novices Use LLM-Based Code Generators to Solve CS1 Coding Tasks in a Self-Paced Learning Environment

Building A Coding Assistant via the Retrieval-Augmented Language Model