Abstract:Due to the critical role of compilers, many compiler testing techniques have been proposed, two most notable categories among which are grammar-based and metamorphic-based techniques. All of them have been extensively studied for testing mature compilers. However, it is typical to develop a new compiler for a new-born programming language in practice. In this scenario, the existing techniques are hardly applicable due to some major reasons: (1) no reference compilers to support differential testing, (2) lack of program analysis tools to support most of metamorphic-based compiler testing, (3) substantial implementation effort incurred by different programming language features. Hence, it is unknown how the existing techniques perform in this new scenario. In this work, we conduct the first exploration (i.e., an industrial case study) to investigate the performance of the existing techniques in this new scenario with substantial adaptations. We adapted grammar-based compiler testing to this scenario by synthesizing new test programs based on code snippets and using compilation crash as test oracle due to the lack of reference compilers for differential testing. We also adapted metamorphic-based compiler testing to this scenario by constructing equivalent test programs under any inputs to relieve the dependence on program analysis tools. We call the adapted techniques SynFuzz and MetaFuzz, respectively. We evaluated both SynFuzz and MetaFuzz on two versions of a new compiler for a new-born programming language in a global IT company. By comparing with the testing practice adopted by the testing team and the general fuzzer (AFL), SynFuzz can detect more bugs during the same testing time, and both SynFuzz and MetaFuzz can complement the other two techniques. In particular, SynFuzz and MetaFuzz have detected 11 previously unknown bugs, all of which have been fixed by the developers. From the industrial case study, we summarized a series of lessons and suggestions for practical use and future research.

A compile method supporting multi-language of control engineering

Compilation approach for dynamically reconfigurable architecture based on grouped tasks

A study of compiler techniques for multiple targets in compiler infrastructures

The Principle and Design of Retargetable Compiler for Multiple Architectures

Realization of Control Algorithm Editor Based on IEC61131-3

Compiler testing: a systematic literature analysis

Advance in Compiler Infrastructures

Research on Programming Language's Syncretic Technology Based on Embedded Component

Retargetable Cross Compilation Techniques

An industrial case study in compiler testing (tool demo)

Testing the Compiler for a New-Born Programming Language: An Industrial Case Study (Experience Paper)

MCompiler: A Synergistic Compilation Framework

Retargetable Cross Compilation Techniques: Comparison and Analysis of GCC and Zephyr.

Compiler backend generation for application specific instruction set processors

Compiling CIL Rewriting Language for Multiprocessors

Architecture Descriptions for the Building of Multi-target Compiling Systems

Modular and Didactic Compiler Design with XML Inter-Phases Communication

The Optimizing Compiler Generator for Embedded Systems

Preliminary Study on the Java-Based Case Teaching Method for Principles of Compiler Design

A Systematic Translation Validation Framework for MLIR-Based Compilers