Abstract:Code Language Models (codeLMs) and Graph Neural Networks (GNNs) are widely used in code vulnerability detection. However, a critical yet often overlooked issue is that GNNs primarily rely on aggregating information from adjacent nodes, limiting structural information transfer to single-layer updates. In code graphs, nodes and relationships typically require cross-layer information propagation to fully capture complex program logic and potential vulnerability patterns. Furthermore, while some studies utilize codeLMs to supplement GNNs with code semantic information, existing integration methods have not fully explored the potential of their collaborative effects. To address these challenges, we introduce Vul-LMGNNs that integrates pre-trained CodeLMs with GNNs, leveraging knowledge distillation to facilitate cross-layer propagation of both code semantic knowledge and structural information. Specifically, Vul-LMGNNs utilizes Code Property Graphs (CPGs) to incorporate code syntax, control flow, and data dependencies, while employing gated GNNs to extract structural information in the CPG. To achieve cross-layer information transmission, we implement an online knowledge distillation (KD) program that enables a single student GNN to acquire structural information extracted from a simultaneously trained counterpart through an alternating training procedure. Additionally, we leverage pre-trained CodeLMs to extract semantic features from code sequences. Finally, we propose an "implicit-explicit" joint training framework to better leverage the strengths of both CodeLMs and GNNs. In the implicit phase, we utilize CodeLMs to initialize the node embeddings of each student GNN. Through online knowledge distillation, we facilitate the propagation of both code semantics and structural information across layers. In the explicit phase, we perform linear interpolation between the CodeLM and the distilled GNN to learn a late fusion model. The proposed method, evaluated across four real-world vulnerability datasets, demonstrated superior performance compared to 17 state-of-the-art approaches. Our source code can be accessed via GitHub: https: //github.com/Vul-LMGNN/vul-LMGGNN.

Enhancing Large Language Models for Secure Code Generation: A Dataset-driven Study on Vulnerability Mitigation

Is Your AI-Generated Code Really Safe? Evaluating Large Language Models on Secure Code Generation with CodeSecEval

An Exploratory Study on Fine-Tuning Large Language Models for Secure Code Generation

CoSec: On-the-Fly Security Hardening of Code LLMs Via Supervised Co-Decoding

Large Language Models for Code: Security Hardening and Adversarial Testing

SecCoder: Towards Generalizable and Robust Secure Code Generation

CodeLMSec Benchmark: Systematically Evaluating and Finding Security Vulnerabilities in Black-Box Code Language Models

Can We Trust Large Language Models Generated Code? A Framework for In-Context Learning, Security Patterns, and Code Evaluations Across Diverse LLMs

ProSec: Fortifying Code LLMs with Proactive Security Alignment

Investigating Large Language Models for Code Vulnerability Detection: An Experimental Study

Fine Tuning Large Language Model for Secure Code Generation

Imperfect Code Generation: Uncovering Weaknesses in Automatic Code Generation by Large Language Models

Vul-LMGNNs: Fusing Language Models and Online-Distilled Graph Neural Networks for Code Vulnerability Detection

Large Language Models and Code Security: A Systematic Literature Review

Outside the Comfort Zone: Analysing LLM Capabilities in Software Vulnerability Detection

Harnessing Large Language Models for Software Vulnerability Detection: A Comprehensive Benchmarking Study

Understanding the Effectiveness of Large Language Models in Detecting Security Vulnerabilities

Large Language Models for Secure Code Assessment: A Multi-Language Empirical Study

How Well Do Large Language Models Serve as End-to-End Secure Code Producers?

Security of Language Models for Code: A Systematic Literature Review

Towards Effectively Detecting and Explaining Vulnerabilities Using Large Language Models