Abstract:Utilizing large language models to generate codes has shown promising meaning in software development revolution. Despite the intelligence shown by the general large language models, their specificity in code generation can still be improved due to the syntactic gap and mismatched vocabulary existing among natural language and different programming languages. In this paper, we propose CodeGRAG, a Graphical Retrieval Augmented Code Generation framework to enhance the performance of LLMs. CodeGRAG builds the graphical view of code blocks based on the control flow and data flow of them to fill the gap between programming languages and natural language, which can facilitate natural language based LLMs for better understanding of code syntax and serve as a bridge among different programming languages. To take the extracted structural knowledge into the foundation models, we propose 1) a hard meta-graph prompt template to transform the challenging graphical representation into informative knowledge for tuning-free models and 2) a soft prompting technique that injects the domain knowledge of programming languages into the model parameters via finetuning the models with the help of a pretrained GNN expert model. Various experiments and ablations are done on four datasets including both the C++ and python languages to validate the hard meta-graph prompt, the soft prompting technique, and the effectiveness of the objectives for pretrained GNN expert. CodeGRAG improves the code generation ability of LLMs and can even offer performance gain for cross-lingual code generation. The implementation is available at <a class="link-external link-https" href="https://anonymous.4open.science/r/Code-5970/" rel="external noopener nofollow">this https URL</a>.

Automatic Code Annotation Generation Based on Multi-dimensional Heterogeneous Graph Structure

Automatic Code Annotation Generation Based on Heterogeneous Graph Structure

Improved Code Summarization via a Graph Neural Network

From Code to Natural Language: Type-Aware Sketch-Based Seq2Seq Learning

Code Generation with Hybrid of Structural and Semantic Features Retrieval

Heterogeneous Directed Hypergraph Neural Network over abstract syntax tree (AST) for Code Classification

Code comment generation based on graph neural network enhanced transformer model for code understanding in open-source software ecosystems

Neural Comment Generation for Source Code with Auxiliary Code Classification Task

Using Graph Neural Network to Analyse and Detect Annotation Misuse in Java Code

GA-SCS: Graph-Augmented Source Code Summarization

Code2Text: Dual Attention Syntax Annotation Networks for Structure-Aware Code Translation

Automatic Source Code Summarization with Graph Attention Networks

CodeGRAG: Bridging the Gap between Natural Language and Programming Language via Graphical Retrieval Augmented Generation

Automatic Generation of Text Descriptive Comments for Code Blocks

A Graph Sequence Neural Architecture for Code Completion with Semantic Structure Features

Automatic Source Code Summarization with Extended Tree-LSTM

The Source Code Comment Generation Based on Deep Reinforcement Learning and Hierarchical Attention

Learning to Represent Programs with Heterogeneous Graphs

Image Annotation Based on Semantic Structure and Graph Learning

Heterogeneous Graph Transformer for Meta-structure Learning with Application in Text Classification

DG-Trans: Automatic Code Summarization Via Dynamic Graph Attention-based Transformer