Abstract:Background: The C and C++ languages hold significant importance in Software Engineering research because of their widespread use in practice. Numerous studies have utilized Machine Learning (ML) and Deep Learning (DL) techniques to detect software vulnerabilities (SVs) in the source code written in these languages. However, the application of these techniques in function-level SV assessment has been largely unexplored. SV assessment is increasingly crucial as it provides detailed information on the exploitability, impacts, and severity of security defects, thereby aiding in their prioritization and remediation. Aims: We conduct the first empirical study to investigate and compare the performance of ML and DL models, many of which have been used for SV detection, for function-level SV assessment in C/C++. Method: Using 9,993 vulnerable C/C++ functions, we evaluated the performance of six multi-class ML models and five multi-class DL models for the SV assessment at the function level based on the Common Vulnerability Scoring System (CVSS). We further explore multi-task learning, which can leverage common vulnerable code to predict all SV assessment outputs simultaneously in a single model, and compare the effectiveness and efficiency of this model type with those of the original multi-class models. Results: We show that ML has matching or even better performance compared to the multi-class DL models for function-level SV assessment with significantly less training time. Employing multi-task learning allows the DL models to perform significantly better, with an average of 8-22% increase in Matthews Correlation Coefficient (MCC). Conclusions: We distill the practices of using data-driven techniques for function-level SV assessment in C/C++, including the use of multi-task DL to balance efficiency and effectiveness. This can establish a strong foundation for future work in this area.

ROMEO: Exploring Juliet through the Lens of Assembly Language

Enhancing Reverse Engineering: Investigating and Benchmarking Large Language Models for Vulnerability Analysis in Decompiled Binaries

Detection of Software Security Weaknesses Using Cross-Language Source Code Representation (CLaSCoRe)

Assemblage: Automatic Binary Dataset Construction for Machine Learning

Is Function Similarity Over-Engineered? Building a Benchmark

Deep-Learning-based Vulnerability Detection in Binary Executables

VulANalyzeR: Explainable Binary Vulnerability Detection with Multi-task Learning and Attentional Graph Convolution

UniBin: Assembly Semantic-enhanced Binary Vulnerability Detection without Disassembly

h4rm3l: A Dynamic Benchmark of Composable Jailbreak Attacks for LLM Safety Assessment

Automated software vulnerability detection with machine learning

Arbiter: Bridging the Static and Dynamic Divide in Vulnerability Discovery on Binary Programs

Automated Code-centric Software Vulnerability Assessment: How Far Are We? An Empirical Study in C/C++

VYPER: Vulnerability detection in binary code

VulDetectBench: Evaluating the Deep Capability of Vulnerability Detection with Large Language Models

How secure is AI-generated Code: A Large-Scale Comparison of Large Language Models

Representation vs. Model: What Matters Most for Source Code Vulnerability Detection

Asteria: Deep Learning-based AST-Encoding for Cross-platform Binary Code Similarity Detection

Software Ethology: An Accurate, Resilient, and Cross-Architecture Binary Analysis Framework

Outside the Comfort Zone: Analysing LLM Capabilities in Software Vulnerability Detection

HAformer: Semantic Fusion of Hex Machine Code and Assembly Code for Cross-Architecture Binary Vulnerability Detection

On the Use of Fine-grained Vulnerable Code Statements for Software Vulnerability Assessment Models