Abstract:Compilers are a kind of important software, and similar to the quality assurance of other software, compiler testing is one of the most widely-used ways of guaranteeing their quality. Compiler bugs tend to occur in compiler optimizations. Detecting optimization bugs needs to consider two main factors: 1) the optimization flags controlling the accessability of the compiler buggy code should be turned on; and 2) the test program should be able to trigger the buggy code. However, existing compiler testing approaches only consider the latter to generate effective test programs, but just run them under several pre-defined optimization levels (e.g., -O0 , -O1 , -O2 , -O3 , -Os in GCC). To better understand the influence of compiler optimizations on compiler testing, we conduct the first empirical study, and find that 1) all the bugs detected under the widely-used optimization levels are also detected under the explored optimization settings (we call a combination of optimization flags turned on for compilation an optimization setting ), while 83.54% of bugs are only detected under the latter; 2) there exist both inhibition effect and promotion effect among optimization flags for compiler testing, indicating the necessity and challenges of considering the factor of compiler optimizations in compiler testing. We then propose the first approach, called COTest , by considering both factors to test compilers. Specifically, COTest first adopts machine learning (the XGBoost algorithm) to model the relationship between test programs and optimization settings, to predict the bug-triggering probability of a test program under an optimization setting. Then, it designs a diversity augmentation strategy to select a set of diverse candidate optimization settings for prediction for a test program. Finally, Top-K optimization settings are selected for compiler testing according to the predicted bug-triggering probabilities. The experiments on GCC and LLVM demonstrate its effectiveness, especially COTest detects 17 previously unknown bugs, 11 of which have been fixed or confirmed by developers.

Enhancing Black-box Compiler Option Fuzzing with LLM Through Command Feedback

A Survey of Modern Compiler Fuzzing

MorFuzz: Fuzzing Processor Via Runtime Instruction Morphing Enhanced Synchronizable Co-simulation.

WhiteFox: White-Box Compiler Fuzzing Empowered by Large Language Models

FuzzCoder: Byte-level Fuzzing Test via Large Language Model

Boosting Compiler Testing via Compiler Optimization Exploration

Fuzzing Deep Learning Compilers with HirGen

Fuzz4All: Universal Fuzzing with Large Language Models

Improving Grey-Box Fuzzing by Modeling Program Behavior

Isolating Compiler Optimization Faults Via Differentiating Finer-grained Options

Isolating Compiler Bugs by Generating Effective Witness Programs with Large Language Models

Fixing Security Vulnerabilities with AI in OSS-Fuzz

A Lightweight and High-Precision Approach for Bulky JavaScript Engines Fuzzing

ProphetFuzz: Fully Automated Prediction and Fuzzing of High-Risk Option Combinations with Only Documentation via Large Language Model

NNSmith: Generating Diverse and Valid Test Cases for Deep Learning Compilers

Compiler Bug Isolation Via Enhanced Test Program Mutation

$μ$AFL: Non-intrusive Feedback-driven Fuzzing for Microcontroller Firmware

Fuzzing with Quantitative and Adaptive Hot-Bytes Identification

Target Program Select Test Input Trim Test Input Mutate Test Input Execute Program Update Queue Instrument Program Grammar

Steelix: program-state based binary fuzzing