Abstract:Source code representation learning is the basis of applying artificial intelligence to many software engineering tasks such as code clone detection, algorithm classification, and code summarization. Recently, many works have tried to improve the performance of source code representation from various perspectives, e.g., introducing the structural information of programs into latent representation. However, when dealing with rapidly expanded unlabeled cross-language source code datasets from the Internet, there are still two issues. Firstly, deep learning models for many code-specific tasks still suffer from the lack of high-quality labels. Secondly, the structural differences among programming languages make it more difficult to process multiple languages in a single neural architecture. To address these issues, in this article, we propose a novel Cross -language Code representation with a large-scale pre-training ( XCode ) method. Concretely, we propose to use several abstract syntax trees and ELMo-enhanced variational autoencoders to obtain multiple pre-trained source code language models trained on about 1.5 million code snippets. To fully utilize the knowledge across programming languages, we further propose a Shared Encoder-Decoder (SED) architecture which uses the multi-teacher single-student method to transfer knowledge from the aforementioned pre-trained models to the distilled SED. The pre-trained models and SED will cooperate to better represent the source code. For evaluation, we examine our approach on three typical downstream cross-language tasks, i.e., source code translation, code clone detection, and code-to-code search, on a real-world dataset composed of programming exercises with multiple solutions. Experimental results demonstrate the effectiveness of our proposed approach on cross-language code representations. Meanwhile, our approach performs significantly better than several code representation baselines on different downstream tasks in terms of multiple automatic evaluation metrics.

Machine Learning Based Source Code Classification Using Syntax Oriented Features

SCC: Automatic Classification of Code Snippets

A Language-Agnostic Model for Semantic Source Code Labeling

Machine Learning-Powered Identification of Source Code Vulnerabilities

Analysis of Program Representations Based on Abstract Syntax Trees and Higher-Order Markov Chains for Source Code Classification Task

Automatic Detection of LLM-generated Code: A Case Study of Claude 3 Haiku

A Survey on Machine Learning Techniques for Source Code Analysis

Predicting the Programming Language of Questions and Snippets of StackOverflow Using Natural Language Processing

Language To Code With Open Source Software

Utilizing Source Code Syntax Patterns to Detect Bug Inducing Commits using Machine Learning Models

Unified Abstract Syntax Tree Representation Learning for Cross-Language Program Classification

A Survey of Machine Learning for Big Code and Naturalness

Deep Learning for Source Code Modeling and Generation

Deep Learning for Source Code Modeling and Generation: Models, Applications and Challenges

Programmers' De-Anonymization Using a Hybrid Approach of Abstract Syntax Tree and Deep Learning

XCode: Towards Cross-Language Code Representation with Large-Scale Pre-Training

PLangRec : Deep-learning model to predict the programming language from a single line of code

A Hybrid Method for Detecting Source-code Plagiarism in Computer Programming Courses

Sparse Attention-Based Neural Networks for Code Classification

Source Code Authorship Attribution Using Hybrid Approach Of Program Dependence Graph And Deep Learning Model

An Empirical Study on Automatically Detecting AI-Generated Source Code: How Far Are We?