Abstract:Source code representation learning is the basis of applying artificial intelligence to many software engineering tasks such as code clone detection, algorithm classification, and code summarization. Recently, many works have tried to improve the performance of source code representation from various perspectives, e.g., introducing the structural information of programs into latent representation. However, when dealing with rapidly expanded unlabeled cross-language source code datasets from the Internet, there are still two issues. Firstly, deep learning models for many code-specific tasks still suffer from the lack of high-quality labels. Secondly, the structural differences among programming languages make it more difficult to process multiple languages in a single neural architecture. To address these issues, in this article, we propose a novel Cross -language Code representation with a large-scale pre-training ( XCode ) method. Concretely, we propose to use several abstract syntax trees and ELMo-enhanced variational autoencoders to obtain multiple pre-trained source code language models trained on about 1.5 million code snippets. To fully utilize the knowledge across programming languages, we further propose a Shared Encoder-Decoder (SED) architecture which uses the multi-teacher single-student method to transfer knowledge from the aforementioned pre-trained models to the distilled SED. The pre-trained models and SED will cooperate to better represent the source code. For evaluation, we examine our approach on three typical downstream cross-language tasks, i.e., source code translation, code clone detection, and code-to-code search, on a real-world dataset composed of programming exercises with multiple solutions. Experimental results demonstrate the effectiveness of our proposed approach on cross-language code representations. Meanwhile, our approach performs significantly better than several code representation baselines on different downstream tasks in terms of multiple automatic evaluation metrics.

Cobol2Vec: Learning Representations of Cobol code

Neural Code Comprehension: A Learnable Representation of Code Semantics

code2vec: Learning Distributed Representations of Code

Import2vec - Learning Embeddings for Software Libraries

Voxel2vec: A Natural Language Processing Approach to Learning Distributed Representations for Scientific Data.

Voxel2vec: A Natural Language Processing Approach to Learning Distributed Representations for Scientific Data

Learning Semantic Vector Representations of Source Code via a Siamese Neural Network

Commit2Vec: Learning Distributed Representations of Code Changes

Hierarchical learning of cross-language mappings through distributed vector representations for code

GraphCode2Vec: Generic Code Embedding via Lexical and Program Dependence Analyses

Towards Demystifying Dimensions of Source Code Embeddings

Towards No-Code Programming of Cobots: Experiments with Code Synthesis by Large Code Models for Conversational Programming

Code Representation Learning At Scale

Enabling Cognitive Intelligence Queries in Relational Databases using Low-dimensional Word Embeddings

Disentangled Code Representation Learning for Multiple Programming Languages.

CompCodeVet: A Compiler-guided Validation and Enhancement Approach for Code Dataset

Open Vocabulary Learning on Source Code with a Graph-Structured Cache

XCode: Towards Cross-Language Code Representation with Large-Scale Pre-Training

COIL: Constrained Optimization in Learned Latent Space: Learning Representations for Valid Solutions

Neuro-symbolic Zero-Shot Code Cloning with Cross-Language Intermediate Representation

sem2vec : Semantics-Aware Assembly Tracelet Embedding