Abstract:Programming language understanding and representation (a.k.a code representation learning) has always been a hot and challenging task in software engineering. It aims to apply deep learning techniques to produce numerical representations of the source code features while preserving its semantics. These representations can be used for facilitating subsequent code-related tasks. The abstract syntax tree (AST), a fundamental code feature, illustrates the syntactic information of the source code and has been widely used in code representation learning. However, there is still a lack of systematic and quantitative evaluation of how well AST-based code representation facilitates subsequent code-related tasks. In this paper, we first conduct a comprehensive empirical study to explore the effectiveness of the AST-based code representation in facilitating follow-up code-related tasks. To do so, we compare the performance of models trained with code token sequence (Token for short) based code representation and AST-based code representation on three popular types of code-related tasks. Surprisingly, the overall quantitative statistical results demonstrate that models trained with AST-based code representation consistently perform worse across all three tasks compared to models trained with Token-based code representation. Our further quantitative analysis reveals that models trained with AST-based code representation outperform models trained with Token-based code representation in certain subsets of samples across all three tasks. We also conduct comprehensive experiments to evaluate and reveal the impact of the choice of AST parsing/preprocessing/encoding methods on AST-based code representation and subsequent code-related tasks. Our study provides future researchers with detailed guidance on how to select solutions at each stage to fully exploit AST.

Code Representation Learning Using Prüfer Sequences (Student Abstract)

Abstract Syntax Tree for Programming Language Understanding and Representation: How Far Are We?

A Novel Neural Source Code Representation Based on Abstract Syntax Tree.

code2seq: Generating Sequences from Structured Representations of Code

Code Representation Learning At Scale

Language-Agnostic Representation Learning of Source Code from Structure and Context

PRISE: LLM-Style Sequence Compression for Learning Temporal Action Abstractions in Control

Assessing the Effectiveness of Syntactic Structure to Learn Code Edit Representations

Hierarchical semantic-aware neural code representation

SPT-Code: Sequence-to-Sequence Pre-Training for Learning Source Code Representations

Represent Code As Action Sequence for Predicting Next Method Call

CoCoAST: Representing Source Code via Hierarchical Splitting and Reconstruction of Abstract Syntax Trees

From Code to Natural Language: Type-Aware Sketch-Based Seq2Seq Learning

Code Representation Pre-training with Complements from Program Executions

A Multi-Task Representation Learning Approach for Source Code

Learning Program Representations with a Tree-Structured Transformer

Universal Representation for Code

TransformCode: A Contrastive Learning Framework for Code Embedding Via Subtree Transformation

Modular Tree Network for Source Code Representation Learning

Integrating Non-Fourier and AST-Structural Relative Position Representations Into Transformer-Based Model for Source Code Summarization

Sequence-to-Sequence Learning Via Shared Latent Representation.