Abstract:Compiler technologies in deep learning and domain-specific hardware acceleration are increasingly adopting extensible compiler frameworks such as Multi-Level Intermediate Representation (MLIR) to facilitate more efficient development. With MLIR, compiler developers can easily define their own custom IRs in the form of MLIR dialects. However, the diversity and rapid evolution of such custom IRs make it impractical to manually write a custom test generator for each dialect. To address this problem, we design a new test generator called SYNTHFUZZ that combines grammar-based fuzzing with custom mutation synthesis. The key essence of SYNTHFUZZ is two fold: (1) It automatically infers parameterized context-dependent custom mutations from existing test cases. (2) It then concretizes the mutation's content depending on the target context and reduces the chance of inserting invalid edits by performing k-ancestor and pre(post)fix matching. SYNTHFUZZ obviates the need to manually define custom mutation operators for each dialect. We compare SYNTHFUZZ to three baselines: Grammarinator, MLIRSmith, and NeuRI. We conduct this comprehensive comparison on four different MLIR projects. Each project defines a new set of MLIR dialects where manually writing a custom test generator would take weeks of effort. Our evaluation shows that SYNTHFUZZ on average improves MLIR dialect pair coverage by 1.75 times, which increases branch coverage by 1.22 times. Further, we show that our context dependent custom mutation increases the proportion of valid tests by up to 1.11 times, indicating that SYNTHFUZZ correctly concretizes its parameterized mutations with respect to the target context. Parameterization of the mutations reduces the fraction of tests violating the base MLIR constraints by 0.57 times, increasing the time spent fuzzing dialect-specific code.

Reinforcement learning guided fuzz testing for a browser's HTML rendering engine

DFL : A DOM sample generation oriented fuzzing framework for browser rendering engines

FA-Fuzz: A Novel Scheduling Scheme Using Firefly Algorithm for Mutation-Based Fuzzing

Reinforcement Learning-Based Fuzzing Technology.

CovRL: Fuzzing JavaScript Engines with Coverage-Guided Reinforcement Learning for LLM-based Mutation

DRLFCfuzzer: fuzzing with Deep-Reinforcement-Learning under Format Constraints

Graph-Based Fuzz Testing for Deep Learning Inference Engine

Graph-based Fuzz Testing for Deep Learning Inference Engines

Coverage-guided fuzzing for deep reinforcement learning systems

Fuzzing JavaScript Engines with a Syntax-Aware Neural Program Model

Deep Learning-Based Hybrid Fuzz Testing

A Lightweight and High-Precision Approach for Bulky JavaScript Engines Fuzzing

High-performance Directional Fuzzing Scheme Based on Deep Reinforcement Learning

Generation-based Fuzzing? Don’t Build a New Generator, Reuse!

JIT-Picking: Differential Fuzzing of JavaScript Engines

Format-aware Learn&Fuzz: Deep Test Data Generation for Efficient Fuzzing

CKGFuzzer: LLM-Based Fuzz Driver Generation Enhanced By Code Knowledge Graph

Mutation-Based Deep Learning Framework Testing Method in JavaScript Environment

Fuzzing Deep Learning Compilers with HirGen

Fuzzing MLIR Compilers with Custom Mutation Synthesis

Evaluating seed selection for fuzzing JavaScript engines