Competition Report: CHC-COMP-21

Grigory Fedyukovich,Philipp Rümmer
DOI: https://doi.org/10.4204/EPTCS.344.7
2021-09-10
Abstract:CHC-COMP-21 is the fourth competition of solvers for Constrained Horn Clauses. In this year, 7 solvers participated at the competition, and were evaluated in 7 separate tracks on problems in linear integer arithmetic, linear real arithmetic, arrays, and algebraic data-types. The competition was run in March 2021 using the StarExec computing cluster. This report gives an overview of the competition design, explains the organisation of the competition, and presents the competition results.
Logic in Computer Science,Symbolic Computation
What problem does this paper attempt to address?
The problem that this paper attempts to solve is to evaluate and compare the differences in performance and effectiveness of the latest tools for solving Constrained Horn Clauses (CHC). Specifically, the CHC - COMP - 21 competition aims to evaluate the capabilities of these tools by using publicly available benchmarks in the real world. The following is the main problem description of this paper: ### 1. **Research Background and Motivation** Constrained Horn Clauses (CHC) is a unified framework for reasoning about software security and has gradually become an effective tool for verifying different aspects of software security in the past decade. CHC can be used to represent various types of systems (such as sequential code, programs with functions and procedures, concurrent programs, or reactive systems) and supports multiple verification methods (such as state invariants, contract - based verification, Owicki - Gries - style invariants, or rely - guarantee methods). Therefore, CHC solvers can be used as off - the - shelf backend modules in verification tools, thereby enabling the construction of modular verification systems. ### 2. **Competition Goals** CHC - COMP - 21 is the fourth CHC solver competition, and its main goals are: - **Compare the performance and effectiveness of existing CHC solvers**: Ensure comprehensive coverage of different types of CHC problems by conducting evaluations on multiple tracks. - **Promote the development of CHC solvers**: By providing a fair competition platform, encourage researchers to improve existing solvers or develop new solvers. - **Promote community cooperation**: By sharing benchmark sets and evaluation results, promote cooperation and communication between academia and industry. ### 3. **Specific Problems** The specific problems involved in the competition include but are not limited to: - **Linear Integer Arithmetic (LIA)**: Handle benchmarks containing linear and non - linear clauses. - **Linear Real Arithmetic (LRA)**: Handle benchmarks encoding transition systems. - **Theory of Arrays**: Handle benchmarks involving array operations. - **Algebraic Data Types (ADT)**: Handle benchmarks involving algebraic data types. ### 4. **Competition Design and Organization** The competition designed 7 different tracks, and each track evaluated different types of problems. The solvers submitted by the contestants were run on the StarExec computing cluster and ranked according to scoring criteria (such as the number of correct answers, CPU time, etc.). ### Summary The core problem of this paper is to evaluate and compare the performance and effectiveness of the latest CHC solvers on different tracks through the CHC - COMP - 21 competition, so as to promote the development and application of CHC solvers.