Abstract:Compiled software is delivered as executable binary code. Developers write source code to express the software semantics, but the compiler converts it to a binary format that the CPU can directly execute. Therefore, binary code analysis is critical to applications in reverse engineering and computer security tasks where source code is not available. However, unlike source code and natural language that contain rich semantic information, binary code is typically difficult for human engineers to understand and analyze. While existing work uses AI models to assist source code analysis, few studies have considered binary code. In this paper, we propose a COntrastive learning Model for Binary cOde Analysis, or COMBO, that incorporates source code and comment information into binary code during representation learning. Specifically, we present three components in COMBO: (1) a primary contrastive learning method for cold-start pre-training, (2) a simplex interpolation method to incorporate source code, comments, and binary code, and (3) an intermediate representation learning algorithm to provide binary code embeddings. Finally, we evaluate the effectiveness of the pre-trained representations produced by COMBO using three indicative downstream tasks relating to binary code: algorithmic functionality classification, binary code similarity, and vulnerability detection. Our experimental results show that COMBO facilitates representation learning of binary code visualized by distribution analysis, and improves the performance on all three downstream tasks by 5.45% on average compared to state-of-the-art large-scale language representation models. To the best of our knowledge, COMBO is the first language representation model that incorporates source code, binary code, and comments into contrastive code representation learning and unifies multiple tasks for binary code analysis.

Contrastive Learning of Functionality-Aware Code Embeddings

Dual Path Structural Contrastive Embeddings for Learning Novel Objects

TransformCode: A Contrastive Learning Framework for Code Embedding Via Subtree Transformation

CodeCSE: A Simple Multilingual Model for Code and Comment Sentence Embeddings

SimCLF: A Simple Contrastive Learning Framework for Function-level Binary Embeddings

CodeRetriever: Unimodal and Bimodal Contrastive Learning for Code Search

Soft-Labeled Contrastive Pre-Training for Function-Level Code Representation

COSEA: Convolutional Code Search with Layer-wise Attention

Cross-Modal Contrastive Learning for Code Search

Fun2Vec:a Contrastive Learning Framework of Function-level Representation for Binary

Pre-Training Representations of Binary Code Using Contrastive Learning

CodeRetriever: A Large Scale Contrastive Pre-Training Method for Code Search

Code Representation Learning At Scale

MCodeSearcher: Multi-View Contrastive Learning for Code Search.

Contrastive Learning with Hallucinating Data for Long-Tailed Face Recognition.

CoCoSoDa: Effective Contrastive Learning for Code Search

Code Representation Pre-training with Complements from Program Executions

Adaptive Code Completion with Meta-learning.

Text and Code Embeddings by Contrastive Pre-Training

Weakly-Supervised Text-driven Contrastive Learning for Facial Behavior Understanding

CODE-MVP: Learning to Represent Source Code from Multiple Views with Contrastive Pre-Training