FuzzTheREST: An Intelligent Automated Black-box RESTful API Fuzzer

Tiago Dias,Eva Maia,Isabel Praça

2024-07-19

Abstract:Software's pervasive impact and increasing reliance in the era of digital transformation raise concerns about vulnerabilities, emphasizing the need for software security. Fuzzy testing is a dynamic analysis software testing technique that consists of feeding faulty input data to a System Under Test (SUT) and observing its behavior. Specifically regarding black-box RESTful API testing, recent literature has attempted to automate this technique using heuristics to perform the input search and using the HTTP response status codes for classification. However, most approaches do not keep track of code coverage, which is important to validate the solution. This work introduces a black-box RESTful API fuzzy testing tool that employs Reinforcement Learning (RL) for vulnerability detection. The fuzzer operates via the OpenAPI Specification (OAS) file and a scenarios file, which includes information to communicate with the SUT and the sequences of functionalities to test, respectively. To evaluate its effectiveness, the tool was tested on the Petstore API. The tool found a total of six unique vulnerabilities and achieved 55\% code coverage.

Software Engineering,Artificial Intelligence,Machine Learning

What problem does this paper attempt to address?

The main focus of this paper is how to effectively perform fuzz testing on RESTful architecture-based application programming interfaces (APIs) to detect potential security vulnerabilities. Specifically, the authors developed an automated fuzz testing tool named FuzzTheREST, which leverages Reinforcement Learning (RL) techniques to guide the generation of effective input data for testing, thereby improving the effectiveness and efficiency of fuzz testing.

The main contributions of FuzzTheREST include:
1. **Reinforcement Learning-driven Input Generation**: Using reinforcement learning algorithms to guide the generation of specific types of input data that can effectively trigger abnormal behaviors or vulnerabilities in the system.
2. **Code Coverage Tracking**: Unlike most existing works, FuzzTheREST not only focuses on the number of errors or vulnerabilities discovered but also tracks code coverage metrics to validate the effectiveness of the solution.
3. **Interpretability and Visibility**: In addition to reporting vulnerabilities, FuzzTheREST provides information about the performance of the reinforcement learning agent, increasing the transparency of the entire fuzz testing process.

The paper details the design, implementation, and a case study of applying FuzzTheREST on the Petstore API. Experimental results show that the tool was able to find 6 unique vulnerabilities during the testing process and achieved 55% code coverage. Additionally, the study discusses the limitations of using HTTP status codes as a standard for vulnerability identification and emphasizes the importance of balancing exploration and exploitation in reinforcement learning methods for efficient fuzz testing.

In summary, this research aims to improve the fuzz testing techniques for RESTful APIs by introducing a new method based on reinforcement learning, thereby enhancing the security and reliability of software.