Abstract:To identify security vulnerabilities in Android applications, numerous static application security testing (SAST) tools have been proposed. However, it poses significant challenges to assess their overall performance on diverse vulnerability types. The task is non-trivial and poses considerable challenges. {Firstly, the absence of a unified evaluation platform for defining and describing tools' supported vulnerability types, coupled with the lack of normalization for the intricate and varied reports generated by different tools, significantly adds to the complexity.} Secondly, there is a scarcity of adequate benchmarks, particularly those derived from real-world scenarios. To address these problems, we are the first to propose a unified platform named VulsTotal, supporting various vulnerability types, enabling comprehensive and versatile analysis across diverse SAST tools. Specifically, we begin by meticulously selecting 11 free and open-sourced SAST tools from a pool of 97 existing options, adhering to clearly defined criteria. After that, we invest significant efforts in comprehending the detection rules of each tool, subsequently unifying 67 general/common vulnerability types for {Android} SAST tools. We also redefine and implement a standardized reporting format, ensuring uniformity in presenting results across all tools. Additionally, to mitigate the problem of benchmarks, we conducted a manual analysis of huge amounts of CVEs to construct a new CVE-based benchmark based on our comprehension of Android app vulnerabilities. Leveraging the evaluation platform, which integrates both existing synthetic benchmarks and newly constructed CVE-based benchmarks from this study, we conducted a comprehensive analysis to evaluate and compare these selected tools from various perspectives, such as general vulnerability type coverage, type consistency, tool effectiveness, and time performance.

A Generic Testbed for Security Enhancement Tools

A cross-platform multifunctional testbed for vulnerability attack

Detecting Buffer-Overflow Vulnerabilities in Smart Grid Devices via Automatic Static Analysis

AutoPwn: Artifact-Assisted Heap Exploit Generation for CTF PWN Competitions

Test Method on Code Injection Vulnerabilities of Web Application

Introduction to the Special Issue on Design for Testability and Reliability of Security-aware Hardware

A Comparison of Static Analysis Technology for Intrusion Prevention

An Overview of Penetration Testing

Research on Principle and Defense of Buffer Overflow Attacks

Analysis of Compiler Technology Enhancement ror Buffer Overflow Prevention

HeapTherapy: An Efficient End-to-End Solution against Heap Buffer Overflows

Poster: Unider: Exploit Attack Emulator Armed with State-of-Art Exploit Techniques

Template for Preparation of Manuscripts for Tsinghua Science and Technology

A Comprehensive Study on Static Application Security Testing (SAST) Tools for Android

An Empirical Study on Detecting and Fixing Buffer Overflow Bugs

A Testing System for Application Specific Integrated Circuit of Information Security

Progress in Research on Buffer Overflow Vulnerability Analysis Technologies

Detecting security vulnerabilities with vulnerability nets

AutoExploit: Automatic Exploit Generation with Security Mitigation

Automatically Validating Static Buffer Overflow Warnings based on Guided Symbolic Execution

Automated Software Testing Starting from Static Analysis: Current State of the Art