FuzzEval: Assessing Fuzzers on Generating Context-Sensitive Inputs

S Mahmudul Hasan,Polina Kozyreva,Endadul Hoque

2024-09-19

Abstract:Cryptographic protocols form the backbone of modern security systems, yet vulnerabilities persist within their implementations. Traditional testing techniques, including fuzzing, have struggled to effectively identify vulnerabilities in cryptographic libraries due to their reliance on context-sensitive inputs. This paper presents a comprehensive evaluation of eleven state-of-the-art fuzzers' ability to generate context-sensitive inputs for testing a cryptographic standard, PKCS#1-v1.5, across thirteen implementations. Our study reveals nuanced performance differences among the fuzzers in terms of the validity and diversity of the produced inputs. This investigation underscores the limitations of existing fuzzers in handling context-sensitive inputs. These findings are expected to drive further research and development in this area.

Cryptography and Security

What problem does this paper attempt to address?

### What problem does this paper attempt to solve?

This paper aims to solve the problem of the insufficient ability of existing fuzzing tools (fuzzers) in generating context - sensitive inputs, especially in the application in cryptographic protocol implementations. Specifically:

1. **Background problems**:
- Cryptographic protocols are the core of modern security systems, but there are still vulnerabilities in their implementations.
- Traditional testing techniques, including fuzzing, are not very effective in identifying vulnerabilities in cryptographic libraries, mainly because these tools rely on context - sensitive inputs.

2. **Research motivation**:
- Existing general - purpose fuzzing tools (such as AFL and its variants) perform poorly in detecting vulnerabilities in cryptographic libraries, which raises a crucial question: what specific limitations prevent these fuzzing tools from effectively identifying security vulnerabilities in cryptographic libraries?
- Generating effective context - sensitive inputs is crucial for bypassing strict input validation and discovering hidden vulnerabilities.

3. **Research objectives**:
- Evaluate the ability of 11 existing state - of - the - art fuzzing tools to generate semantically valid context - sensitive inputs, especially for 13 implementations of the PKCS#1 - v1.5 encryption standard.
- Reveal the performance differences of different fuzzing tools in generating valid and diverse inputs, thus providing a basis for improving fuzzing tools.

4. **Research methods**:
- Use a platform named FuzzEval to conduct large - scale experiments, which can evaluate the effectiveness and diversity of fuzzing tools in generating context - sensitive inputs.
- Quantify the performance of different fuzzing tools through a series of metrics (such as the percentage of valid inputs, edit distance, normalized longest common subsequence, etc.).

5. **Expected impact**:
- This research reveals the limitations of existing fuzzing tools in handling context - sensitive inputs and is expected to promote the community to further develop improved general - purpose fuzzing tools.

In summary, this paper attempts to solve the problem of the insufficient ability of existing fuzzing tools in generating context - sensitive inputs in order to improve their effectiveness in detecting vulnerabilities in cryptographic libraries.