Abstract: Large language models (LLMs) have showcased remarkable prowess in code generation. However, automated code generation is still challenging since it requires a high-level semantic mapping between natural language requirements and codes. Most existing LLMs-based approaches for code generation rely on decoder-only causal language models often treate codes merely as plain text tokens \ie feeding the requirements as a prompt input, and outputing code as flat sequence of tokens, potentially missing the rich semantic features inherent in source code. To bridge this gap, this paper proposes the "Semantic Chain-of-Thought" approach to intruduce semantic information of code, named SeCoT. Our motivation is that the semantic information of the source code (\eg data flow and control flow) describes more precise program execution behavior, intention and function. By guiding LLM consider and integrate semantic information, we can achieve a more granular understanding and representation of code, enhancing code generation accuracy. Meanwhile, while traditional techniques leveraging such semantic information require complex static or dynamic code analysis to obtain features such as data flow and control flow, SeCoT demonstrates that this process can be fully automated via the intrinsic capabilities of LLMs (\ie in-context learning), while being generalizable and applicable to challenging domains. While SeCoT can be applied with different LLMs, this paper focuses on the powerful GPT-style models: ChatGPT(close-source model) and WizardCoder(open-source model). The experimental study on three popular DL benchmarks (\ie HumanEval, HumanEval-ET and MBPP) shows that SeCoT can achieves state-of-the-art performance, greatly improving the potential for large models and code generation.

CODEP: Grammatical Seq2Seq Model for General-Purpose Code Generation.

CodePAD: Sequence-based Code Generation with Pushdown Automaton

Execution-based Code Generation using Deep Reinforcement Learning

From Code to Natural Language: Type-Aware Sketch-Based Seq2Seq Learning

Seq2Seq or Seq2Tree: Generating Code Using Both Paradigms Via Mutual Learning.

Fine-grained Pseudo-code Generation Method via Code Feature Extraction and Transformer

GAP-Gen: Guided Automatic Python Code Generation

CodeDPO: Aligning Code Models with Self Generated and Verified Source Code

Outline, Then Details: Syntactically Guided Coarse-To-Fine Code Generation

Automatic Code Generation using Pre-Trained Language Models

Bridging Code Semantic and LLMs: Semantic Chain-of-Thought Prompting for Code Generation

code2seq: Generating Sequences from Structured Representations of Code

Automatic Generation of Pseudocode with Attention Seq2seq Model

Compilable Neural Code Generation with Compiler Feedback

Code Generation Based on Deep Learning: a Brief Review

Learning to Generate Code Sketches.

Planning with Large Language Models for Code Generation

Code Generation with Hybrid of Structural and Semantic Features Retrieval

CodeGRAG: Bridging the Gap between Natural Language and Programming Language via Graphical Retrieval Augmented Generation

NatGen: Generative pre-training by "Naturalizing" source code