Abstract:Compiler optimization decisions are often based on hand-crafted heuristics centered around a few established benchmark suites. Alternatively, they can be learned from feature and performance data produced during compilation. However, data-driven compiler optimizations based on machine learning models require large sets of quality data for training in order to match or even outperform existing human-crafted heuristics. In static compilation setups, related work has addressed this problem with iterative compilation. However, a dynamic compiler may produce different data depending on dynamically-chosen compilation strategies, which aggravates the generation of comparable data. We propose compilation forking, a technique for generating consistent feature and performance data from arbitrary, dynamically-compiled programs. Different versions of program parts with the same profiling and compilation history are executed within single program runs to minimize noise stemming from dynamic compilation and the runtime environment. Our approach facilitates large-scale performance evaluations of compiler optimization decisions. Additionally, compilation forking supports creating domain-specific compilation strategies based on machine learning by providing the data for model training. We implemented compilation forking in the GraalVM compiler in a programming-language-agnostic way. To assess the quality of the generated data, we trained several machine learning models to replace compiler heuristics for loop-related optimizations. The trained models perform equally well to the highly-tuned compiler heuristics when comparing the geometric means of benchmark suite performances. Larger impacts on few single benchmarks range from speedups of 20% to slowdowns of 17%. The presented approach can be implemented in any dynamic compiler. We believe that it can help to analyze compilation decisions and leverage the use of machine learning into dynamic compilation.

Machine Learning in Compiler Optimisation

A Survey on Compiler Autotuning using Machine Learning

Accelerating Chip Design with Machine Learning: from Pre-Silicon to Post-Silicon

Towards Compile-Time-Reducing Compiler Optimization Selection via Machine Learning

Feature Mining for Machine Learning Based Compilation Optimization

CompilerDream: Learning a Compiler World Model for General Code Optimization

The Deep Learning Compiler: A Comprehensive Survey

Compiler Autotuning through Multiple Phase Learning

MCompiler: A Synergistic Compilation Framework

Compiler Auto-tuning through Multiple Phase Learning

CMLCompiler: A Unified Compiler for Classical Machine Learning

ACPO: AI-Enabled Compiler-Driven Program Optimization

AI Powered Compiler Techniques for DL Code Optimization

Compilation Optimization Pass Selection Using Gate Graph Attention Neural Network for Reliability Improvement

Deep Data Flow Analysis

Meta Large Language Model Compiler: Foundation Models of Compiler Optimization

Large Language Models for Compiler Optimization

Compilation Forking: A Fast and Flexible Way of Generating Data for Compiler-Internal Machine Learning Tasks

Machine Learning Optimization Techniques: A Survey, Classification, Challenges, and Future Research Issues

Machine Learning-Based Analysis of Program Binaries: A Comprehensive Study

ProGraML: Graph-based Deep Learning for Program Optimization and Analysis