Python Fuzzing for Trustworthy Machine Learning Frameworks

Ilya Yegorov,Eli Kobrin,Darya Parygina,Alexey Vishnyakov,Andrey Fedotov
2024-03-19
Abstract:Ensuring the security and reliability of machine learning frameworks is crucial for building trustworthy AI-based systems. Fuzzing, a popular technique in secure software development lifecycle (SSDLC), can be used to develop secure and robust software. Popular machine learning frameworks such as PyTorch and TensorFlow are complex and written in multiple programming languages including C/C++ and Python. We propose a dynamic analysis pipeline for Python projects using the Sydr-Fuzz toolset. Our pipeline includes fuzzing, corpus minimization, crash triaging, and coverage collection. Crash triaging and severity estimation are important steps to ensure that the most critical vulnerabilities are addressed promptly. Furthermore, the proposed pipeline is integrated in GitLab CI. To identify the most vulnerable parts of the machine learning frameworks, we analyze their potential attack surfaces and develop fuzz targets for PyTorch, TensorFlow, and related projects such as h5py. Applying our dynamic analysis pipeline to these targets, we were able to discover 3 new bugs and propose fixes for them.
Cryptography and Security,Artificial Intelligence,Software Engineering
What problem does this paper attempt to address?
The paper aims to address the security and reliability issues of machine learning frameworks, particularly popular ones like PyTorch and TensorFlow. The authors propose a dynamic analysis pipeline that utilizes the Sydr-Fuzz toolset to perform fuzz testing on Python projects, including steps such as corpus minimization, crash classification, and coverage collection. In this way, the authors are able to identify and fix potential vulnerabilities in these frameworks. Specifically, the paper focuses on: 1. **Fuzz Testing**: A method to discover software vulnerabilities by generating unexpected or malformed data inputs to the program. 2. **Dynamic Analysis Pipeline**: Includes steps such as fuzz testing, corpus minimization, crash classification, and coverage collection. 3. **Security Enhancement**: Improving the security of frameworks like PyTorch, TensorFlow, and their related projects through fuzz testing. 4. **Continuous Integration**: Automating the fuzz testing process and integrating it with GitLab CI to continuously monitor and improve code quality. The authors validated the effectiveness of their method through experiments, discovering several new vulnerabilities and proposing fixes. Additionally, they developed an automated crash classification system called Casr to simplify the analysis process of crash reports.