Abstract:Code completion plays a prominent role in modern integrated development environments (IDEs). Machine learning has become ubiquitous in analogous natural language writing and search software, surfacing more relevant autocompletions and search suggestions in fewer keystrokes. Prior research has reported training high-accuracy, deep neural networks for modeling source code, but little attention has been given to the practical constraints imposed by interactive developer tools. In particular, neural language models for source code modeling like the one described in Maybe Deep Neural Networks are the Best Choice for Modeling Source Code are framed around code completion, but only report accuracy of next-token prediction. However, in order for a language model (LM) to work well within real-world code completion systems, it must also always make suggestions that produce valid code that typechecks to support code completion's role in correctness-checking; return instantaneous results to help programmers code more efficiently in fewer keystrokes; and be small enough to fit comfortably on disk and in memory on developer workstations, since virtually all modern IDEs run locally and support offline usage. To meet these additional requirements, we propose a novel design for predicting top-k next tokens that combines static analysis' ability to enumerate all valid keywords and in-scope identifiers with the ability of a language model to place a probability distribution over them. Our model mixes character-level input representation with token output to represent out-of-vocabulary (OOV) tokens meaningfully and minimize prediction latency. OOV tokens can be predicted through detection of local repetition common in software. This design achieves state-of-art accuracy in source code modeling and fits the constraints imposed by real-world code completion implementations in modern IDEs.

CCMC: Code Completion with a Memory Mechanism and a Copy Mechanism

A Self-Attentional Neural Architecture for Code Completion with Multi-Task Learning.

Prompt-based Code Completion via Multi-Retrieval Augmented Generation

Multi-task learning based pre-trained language model for code completion

Improve Language Modeling for Code Completion Through Learning General Token Repetition of Source Code with Optimized Memory

LongCoder: A Long-Range Pre-trained Language Model for Code Completion

ReACC: A Retrieval-Augmented Code Completion Framework

CoCoMIC: Code Completion By Jointly Modeling In-file and Cross-file Context

MemoNet: Memorizing All Cross Features' Representations Efficiently via Multi-Hash Codebook Network for CTR Prediction

R2C2-Coder: Enhancing and Benchmarking Real-world Repository-level Code Completion Abilities of Code Large Language Models

A unified multi-task learning model for AST-level and token-level code completion

Improving AST-Level Code Completion with Graph Retrieval and Multi-Field Attention

GraphCoder: Enhancing Repository-Level Code Completion via Code Context Graph-based Retrieval and Language Model

Bridge and Hint: Extending Pre-trained Language Models for Long-Range Code

Sequence Model Design for Code Completion in the Modern IDE

Towards Full-line Code Completion with Neural Language Models

Deep-AutoCoder: Learning to Complete Code Precisely with Induced Code Tokens

When Neural Code Completion Models Size up the Situation: Attaining Cheaper and Faster Completion through Dynamic Model Inference

A Graph Sequence Neural Architecture for Code Completion with Semantic Structure Features

Adaptive Code Completion with Meta-learning.

Enriching Source Code with Contextual Data for Code Completion Models: An Empirical Study