ProFuzzBench: A Benchmark for Stateful Protocol Fuzzing

Roberto Natella,Van-Thuan Pham

DOI: https://doi.org/10.48550/arXiv.2101.05102

2021-01-13

Abstract:We present a new benchmark (ProFuzzBench) for stateful fuzzing of network protocols. The benchmark includes a suite of representative open-source network servers for popular protocols, and tools to automate experimentation. We discuss challenges and potential directions for future research based on this benchmark.

Cryptography and Security,Networking and Internet Architecture,Software Engineering

What problem does this paper attempt to address?

The paper attempts to address the challenges in network protocol fuzzing, particularly for stateful protocol fuzzing. Specifically, the paper proposes a new benchmarking tool (ProFuzzBench) aimed at providing a standardized testing platform for stateful fuzzing of network protocols.

### Main Issues Include:
1. **Test Coverage**: Existing fuzzing tools struggle to effectively cover all possible state combinations when dealing with stateful protocols.
2. **Experimental Reproducibility**: The lack of standardized benchmarking tools makes it difficult to compare and evaluate different fuzzing techniques.
3. **Configuration Complexity**: The software implementation of network protocols usually requires complex configurations, which increases the difficulty of fuzzing.
4. **Multi-Party Protocols**: Some protocols (e.g., SIP) require multiple parties (e.g., server, caller, and callee), while most existing fuzzing tools only support the client-server model.
5. **Non-Deterministic Execution**: The non-deterministic behavior of network servers (e.g., multi-threading and event-driven I/O) leads to different path coverage for the same input in different runs, affecting the effectiveness of coverage-based fuzzing.
6. **Test Execution Speed**: The initialization process of network servers and network overhead make fuzzing slower, especially when dealing with long message sequences.

### Solutions:
- **Benchmarking Tool**: ProFuzzBench provides a set of representative open-source network server implementations and automated tools for fuzzing experiments.
- **Automation**: Through Docker containerization technology, it achieves automated deployment, configuration, execution, and analysis of target software and fuzzing tools.
- **Patches**: The target software is patched to remove randomness, add message markers, and state encoding to improve test reproducibility and efficiency.
- **Multi-Instance Parallel Execution**: Supports multi-instance parallel execution, utilizing multi-core architecture to improve testing efficiency.
- **Data Analysis**: Provides detailed statistical analysis tools, generating time-series data on code coverage and protocol state coverage, facilitating the comparison of different fuzzing techniques.

Through these measures, ProFuzzBench aims to advance the technology of stateful network protocol fuzzing and provide researchers with a standardized testing platform.