Abstract:Code Language Models (codeLMs) and Graph Neural Networks (GNNs) are widely used in code vulnerability detection. However, a critical yet often overlooked issue is that GNNs primarily rely on aggregating information from adjacent nodes, limiting structural information transfer to single-layer updates. In code graphs, nodes and relationships typically require cross-layer information propagation to fully capture complex program logic and potential vulnerability patterns. Furthermore, while some studies utilize codeLMs to supplement GNNs with code semantic information, existing integration methods have not fully explored the potential of their collaborative effects. To address these challenges, we introduce Vul-LMGNNs that integrates pre-trained CodeLMs with GNNs, leveraging knowledge distillation to facilitate cross-layer propagation of both code semantic knowledge and structural information. Specifically, Vul-LMGNNs utilizes Code Property Graphs (CPGs) to incorporate code syntax, control flow, and data dependencies, while employing gated GNNs to extract structural information in the CPG. To achieve cross-layer information transmission, we implement an online knowledge distillation (KD) program that enables a single student GNN to acquire structural information extracted from a simultaneously trained counterpart through an alternating training procedure. Additionally, we leverage pre-trained CodeLMs to extract semantic features from code sequences. Finally, we propose an "implicit-explicit" joint training framework to better leverage the strengths of both CodeLMs and GNNs. In the implicit phase, we utilize CodeLMs to initialize the node embeddings of each student GNN. Through online knowledge distillation, we facilitate the propagation of both code semantics and structural information across layers. In the explicit phase, we perform linear interpolation between the CodeLM and the distilled GNN to learn a late fusion model. The proposed method, evaluated across four real-world vulnerability datasets, demonstrated superior performance compared to 17 state-of-the-art approaches. Our source code can be accessed via GitHub: https: //github.com/Vul-LMGNN/vul-LMGGNN.

XShellGNN: Cross-file Web Shell Detection Based on Graph Neural Network

Detection Method of Domain Names Generated by DGAs Based on Semantic Representation and Deep Neural Network

Ignnvd: A Novel Software Vulnerability Detection Model Based on Integrated Graph Neural Networks

GlareShell: Graph learning-based PHP webshell detection for web server of industrial internet

Webshell Detection Technology Based on Deep Learning

TACSan: Enhancing Vulnerability Detection with Graph Neural Network

Vul-LMGNNs: Fusing Language Models and Online-Distilled Graph Neural Networks for Code Vulnerability Detection

Use of Graph Neural Networks in Aiding Defensive Cyber Operations

Research and application of artificial intelligence based webshell detection model: A literature review

ShellCore: Automating Malicious IoT Software Detection Using Shell Commands Representation

E-GVD: Efficient Software Vulnerability Detection Techniques Based on Graph Neural Network

ShellCore: Automating Malicious IoT Software Detection by Using Shell Commands Representation

BERT-Embedding-Based JSP Webshell Detection on Bytecode Level Using XGBoost

Identifying Backdoored Graphs in Graph Neural Network Training: An Explanation-Based Approach with Novel Metrics

Graph Neural Backdoor: Fundamentals, Methodologies, Applications, and Future Directions

Cross-site scripting attack detection based on a modified convolution neural network

OCGEC: One-class Graph Embedding Classification for DNN Backdoor Detection

GNN-based Advanced Feature Integration for ICS Anomaly Detection

JStrong: Malicious JavaScript detection based on code semantic representation and graph neural network

An Android Malware Detection Approach to Enhance Node Feature Differences in a Function Call Graph Based on GCNs

A software vulnerability detection method based on deep learning with complex network analysis and subgraph partition