Abstract:Background: The C and C++ languages hold significant importance in Software Engineering research because of their widespread use in practice. Numerous studies have utilized Machine Learning (ML) and Deep Learning (DL) techniques to detect software vulnerabilities (SVs) in the source code written in these languages. However, the application of these techniques in function-level SV assessment has been largely unexplored. SV assessment is increasingly crucial as it provides detailed information on the exploitability, impacts, and severity of security defects, thereby aiding in their prioritization and remediation. Aims: We conduct the first empirical study to investigate and compare the performance of ML and DL models, many of which have been used for SV detection, for function-level SV assessment in C/C++. Method: Using 9,993 vulnerable C/C++ functions, we evaluated the performance of six multi-class ML models and five multi-class DL models for the SV assessment at the function level based on the Common Vulnerability Scoring System (CVSS). We further explore multi-task learning, which can leverage common vulnerable code to predict all SV assessment outputs simultaneously in a single model, and compare the effectiveness and efficiency of this model type with those of the original multi-class models. Results: We show that ML has matching or even better performance compared to the multi-class DL models for function-level SV assessment with significantly less training time. Employing multi-task learning allows the DL models to perform significantly better, with an average of 8-22% increase in Matthews Correlation Coefficient (MCC). Conclusions: We distill the practices of using data-driven techniques for function-level SV assessment in C/C++, including the use of multi-task DL to balance efficiency and effectiveness. This can establish a strong foundation for future work in this area.

Representation vs. Model: What Matters Most for Source Code Vulnerability Detection

Function-Level Vulnerability Detection Through Fusing Multi-Modal Knowledge

A Survey of Source Code Representations for Machine Learning-Based Cybersecurity Tasks

Survey of Source Code Vulnerability Analysis Based on Deep Learning

An extensive study of the effects of different deep learning models on code vulnerability detection in Python code

On the Code Vulnerability Detection Based on Deep Learning: A Comparative Study

Source Code Vulnerability Detection: Combining Code Language Models and Code Property Graphs

Automated Code-centric Software Vulnerability Assessment: How Far Are We? An Empirical Study in C/C++

Code Vulnerability Detection Based on Deep Sequence and Graph Models: A Survey

Vulnerability Detection in C/C++ Code with Deep Learning

Automated software vulnerability detection with machine learning

Automated Vulnerability Detection in Source Code Using Minimum Intermediate Representation Learning

Machine Learning-Powered Identification of Source Code Vulnerabilities

Harnessing the Power of LLMs in Source Code Vulnerability Detection

SAFE: Advancing Large Language Models in Leveraging Semantic and Syntactic Relationships for Software Vulnerability Detection

Explaining the Contributing Factors for Vulnerability Detection in Machine Learning

A Comparative Study of Deep Learning-Based Vulnerability Detection System

Vulnerability Detection with Representation Learning.

DiverseVul: A New Vulnerable Source Code Dataset for Deep Learning Based Vulnerability Detection

Transformer-based Vulnerability Detection in Code at EditTime: Zero-shot, Few-shot, or Fine-tuning?

Outside the Comfort Zone: Analysing LLM Capabilities in Software Vulnerability Detection