Abstract:Virtually any software running on a computer has been processed by a compiler or a compiler-like tool. Because compilers are such a crucial piece of infrastructure for building software, their correctness is of paramount importance. To validate and increase the correctness of compilers, significant research efforts have been devoted to testing compilers. This survey article provides a comprehensive summary of the current state-of-the-art of research on compiler testing. The survey covers different aspects of the compiler testing problem, including how to construct test programs, what test oracles to use for determining whether a compiler behaves correctly, how to execute compiler tests efficiently, and how to help compiler developers take action on bugs discovered by compiler testing. Moreover, we survey work that empirically studies the strengths and weaknesses of current compiler testing research and practice. Based on the discussion of existing work, we outline several open challenges that remain to be addressed in future work.

What problem does this paper attempt to address?

### Problems the paper attempts to solve This paper aims to provide a comprehensive overview of the latest research progress in the field of compiler testing. Specifically, it focuses on the following aspects: 1. **Constructing test programs**: How to generate effective test programs that can cover different parts of the compiler and conform to the language specification. 2. **Selection of test oracles**: How to determine whether the behavior of the compiler is correct, that is, how to judge whether the output of the compiler meets expectations. 3. **Improving test efficiency**: How to efficiently execute compiler tests to reduce test time and resource consumption. 4. **Post - processing of test results**: How to help compiler developers prioritize and understand the errors discovered through testing. 5. **Empirical research**: Conduct an empirical analysis of the strengths and weaknesses of current compiler testing research and practice. 6. **Future challenges**: Based on the discussion of existing research, point out the open problems that need to be solved in future work. ### Background and motivation The compiler is an important tool for building software, and almost all computer programs are processed by a compiler or a similar tool. Due to the central position of the compiler in software development, its correctness is crucial. However, despite a great deal of effort to improve the quality of the compiler, compilers still have various bugs. These bugs may lead to the generation of incorrect binary code from correct source code and may even spread to applications that depend on the compiler, causing unexpected and potentially harmful behavior. ### Main challenges 1. **Lack of formal specifications**: The specific behavior of the compiler often does not have a clear formal specification, especially regarding when to apply which optimizations. 2. **Semantic richness of input and output languages**: It is very difficult to automatically create source code with non - trivial behavior because small input changes may lead to huge output differences. 3. **Multiple configuration options**: Compilers usually support multiple optimization levels, language variants, and target platforms, which makes the test space extremely large and difficult to exhaust. ### Research significance Given the importance of the compiler, it is of great significance to improve its reliability and correctness. Research on compiler testing not only helps to discover and fix bugs in the compiler but also can improve the efficiency of software debugging and reduce the waste of development time caused by compiler errors. ### Conclusion This paper systematically reviews the existing compiler testing methods, summarizes the current research progress, and points out the direction of future research. This is of great value for researchers and practitioners to understand the current situation and challenges of compiler testing.

A Survey of Compiler Testing

Compiler testing: a systematic literature analysis

A Survey of Modern Compiler Fuzzing

An Empirical Comparison Of Compiler Testing Techniques

Testing the Compiler for a New-Born Programming Language: An Industrial Case Study (Experience Paper)

An industrial case study in compiler testing (tool demo)

Compiler test suite: evaluation and use in an automated test environment

SpecTest: Specification-Based Compiler Testing

PTE: Axiomatic Semantics based Compiler Testing

Compiler Testing With Relaxed Memory Models

Learning to accelerate compiler testing

Compiler Bug Isolation Via Enhanced Test Program Mutation

Boosting Compiler Testing via Compiler Optimization Exploration

A survey of combinatorial testing

Boosting Compiler Testing Via Eliminating Test Programs with Long-Execution-Time

Learning to Prioritize Test Programs for Compiler Testing

Validating JIT Compilers via Compilation Space Exploration

A large-scale empirical study of compiler errors in continuous integration.

A Survey on Compiler Autotuning using Machine Learning

History-Guided Configuration Diversification for Compiler Test-Program Generation

Compiler Testing using Template Java Programs