Is Stateful Fuzzing Really Challenging?

Cristian Daniele

2024-06-12

Abstract:Fuzzing has been proven extremely effective in finding vulnerabilities in software. When it comes to fuzz stateless systems, analysts have no doubts about the choice to make. In fact, among the plethora of stateless fuzzers devised in the last 20 years, AFL (with its descendants AFL++ and LibAFL) stood up for its effectiveness, speed and ability to find bugs. On the other hand, when dealing with stateful systems, it is not clear what is the best tool to use. In fact, the research community struggles to devise (and benchmark) effective and generic stateful fuzzers. In this short paper, we discuss the reasons that make stateful fuzzers difficult to devise and benchmark.

Software Engineering,Cryptography and Security

What problem does this paper attempt to address?

The paper primarily explores the challenges faced by stateful fuzzing and the difficulties in its benchmarking. Specifically:

1. **Research Background and Motivation**: Fuzzing techniques are highly effective in discovering software vulnerabilities, especially when dealing with stateless systems. However, when it comes to stateful systems, existing tools and techniques often perform poorly. This paper aims to discuss why designing and benchmarking effective stateful fuzzing tools is so challenging.

2. **Difference Between Stateless and Stateful Fuzzing**: Stateless fuzzing tools (such as AFL and its variants) can achieve good code coverage by mutating a single message. In contrast, stateful fuzzing needs to consider both the content of the messages and their sending order to theoretically achieve 100% coverage. This makes stateless fuzzing tools less effective on stateful systems.

3. **Limitations of Stateless Fuzzing Tools**: The authors detail the main shortcomings of stateless fuzzing tools when dealing with stateful systems, including the inability to transmit messages over a network, the need to restart the System Under Test (SUT) after each message is sent, and the inability to mutate the order of messages.

4. **Strategies for Handling Stateful Systems**: Fuzzing methods for stateful systems can be broadly divided into two categories—methods that bypass stateful characteristics and methods that directly handle stateful characteristics. The former attempts to make stateless fuzzing tools partially applicable to stateful systems through preprocessing or adding loops; the latter achieves comprehensive testing of stateful systems by sending complete interaction sequences.

5. **Benchmarking Challenges of Stateful Fuzzing**: Current stateful fuzzing tools lack a unified evaluation standard, particularly in monitoring state coverage. Additionally, different stateful fuzzers may excel in exploring state models or specific state tests, so benchmarking results may be influenced by the characteristics of the system under test.

In summary, the paper attempts to reveal the core challenges in the field of stateful fuzzing and provides references for future research directions.