Abstract:Vulnerabilities publicly disclosed in the National Vulnerability Database (NVD) are assigned with CVE (Common Vulnerabilities and Exposures) IDs and associated with specific software versions. Many organizations, including IT companies and government, heavily rely on the disclosed vulnerabilities in NVD to mitigate their security risks. Once a software is claimed as vulnerable by NVD, these organizations would examine the presence of the vulnerable versions of the software and assess the impact on themselves. However, the version information about vulnerable software in NVD is not always reliable. Nguyen et al. find that the version information of many CVE vulnerabilities is spurious and propose an approach based on the original SZZ algorithm (i.e., an approach to identify bug-introducing commits) to assess the software versions affected by CVE vulnerabilities. However, SZZ algorithms are designed for common bugs, while vulnerabilities and bugs are different. Many bugs are introduced by a recent bug-fixing commit, but vulnerabilities are usually introduced in their initial versions. Thus, the current SZZ algorithms often fail to identify the inducing commits for vulnerabilities. Therefore, in this study, we propose an approach based on an improved SZZ algorithm to refine software versions affected by CVE vulnerabilities. Our proposed SZZ algorithm leverages the line mapping algorithms to identify the earliest commit that modified the vulnerable lines, and then considers these commits to be the vulnerability-inducing commits, as opposed to the previous SZZ algorithms that assume the commits that last modified the buggy lines as the inducing commits. To evaluate our proposed approach, we manually annotate the true inducing commits and verify the vulnerable versions for 172 CVE vulnerabilities with fixing commits from two publicly available datasets with five C/C++ and 41 Java projects, respectively. We find that 99 out of 172 vulnerabilities whose version information is spurious. The experiment results show that our proposed approach can identify more vulnerabilities with the true inducing commits and correct vulnerable versions than the previous SZZ algorithms. Our approach outperforms the previous SZZ algorithms in terms of F1-score for identifying vulnerability-inducing commits on both C/C++ and Java projects (0.736 and 0.630, respectively). For refining vulnerable versions, our approach also achieves the best performance on the two datasets in terms of F1-score (0.928 and 0.952).

Benchmark Requirements for Assessing Software Security Vulnerability Testing Tools.

Unveil the Mystery of Critical Software Vulnerabilities

Towards More Practical Automation of Vulnerability Assessment

V-SZZ: Automatic Identification of Version Ranges Affected by CVE Vulnerabilities

Automated Software Testing Starting from Static Analysis: Current State of the Art

Evaluating C/C++ Vulnerability Detectability of Query-Based Static Application Security Testing Tools

Benchmarking Software Vulnerability Detection Techniques: A Survey

A Comprehensive Study on Static Application Security Testing (SAST) Tools for Android

How to build a vulnerability benchmark to overcome cyber security attacks

An Empirical Study on Benchmarks of Artificial Software Vulnerabilities

An Extensive Comparison of Static Application Security Testing Tools

A cross-platform multifunctional testbed for vulnerability attack

Static Application Security Testing (SAST) Tools for Smart Contracts: How Far Are We?

Evaluation of Static Vulnerability Detection Tools with Java Cryptographic API Benchmarks

A systematic review of fuzzy testing for information systems and applications

Comprehensive Evaluation of Static Analysis Tools for Their Performance in Finding Vulnerabilities in Java Code

A Survey on Software Vulnerability Exploitability Assessment

Automation of System Security Vulnerabilities Detection Using Open-Source Software

Do I really need all this work to find vulnerabilities? An empirical case study comparing vulnerability detection techniques on a Java application

Finding your way in the testing jungle: a learning approach to web security testing

Security Requirements Engineering in the Wild: A Survey of Common Practices