Quantifying Software Correctness by Combining Architecture Modeling and Formal Program Analysis

Florian Lanzinger,Christian Martin,Frederik Reiche,Samuel Teuber,Robert Heinrich,Alexander Weigl

DOI: https://doi.org/10.1145/3605098.3636008

2024-01-26

Abstract:Most formal methods see the correctness of a software system as a binary decision. However, proving the correctness of complex systems completely is difficult because they are composed of multiple components, usage scenarios, and environments. We present QuAC, a modular approach for quantifying the correctness of service-oriented software systems by combining software architecture modeling with deductive verification. Our approach is based on a model of the service-oriented architecture and the probabilistic usage scenarios of the system. The correctness of a single service is approximated by a coverage region, which is a formula describing which inputs for that service are proven to not lead to an erroneous execution. The coverage regions can be determined by a combination of various analyses, e.g., formal verification, expert estimations, or testing. The coverage regions and the software model are then combined into a probabilistic program. From this, we can compute the probability that under a given usage profile no service is called outside its coverage region. If the coverage region is large enough, then instead of attempting to get 100% coverage, which may be prohibitively expensive, run-time verification or testing approaches may be used to deal with inputs outside the coverage region. We also present an implementation of QuAC for Java using the modeling tool Palladio and the deductive verification tool KeY. We demonstrate its usability by applying it to a software simulation of an energy system.

Software Engineering,Logic in Computer Science

What problem does this paper attempt to address?

The problem this paper attempts to address is how to quantify the correctness of software systems. Most formal methods treat the correctness of software systems as a binary decision problem, i.e., the software is either completely correct or incorrect. However, for complex systems, this binary judgment is overly simplistic because these systems are often composed of multiple components, usage scenarios, and environments, making comprehensive verification difficult. Therefore, the paper proposes a method called Qu ac ("Quantifying Architecture and Code") to quantify the correctness of service-oriented software systems by combining software architecture modeling and formal program analysis.

Specifically, the Qu ac method is based on service-oriented architecture models and probabilistic usage scenarios of the system to approximate the correctness of individual services. It describes which inputs will not cause service execution errors through coverage regions. Coverage regions can be determined by various analysis methods, such as formal verification, expert estimation, or testing. These coverage regions are then combined with the software model to form a probabilistic program, from which the probability that no service is called outside its coverage region under a given usage profile can be calculated. This method not only addresses the multi-component problem of complex systems but also provides fine-grained quantitative feedback to assess system reliability.

Moreover, the Qu ac method can be applied at different stages of software development. In the early stages, developers can estimate coverage regions, and in the later stages, these estimates can be replaced with testing and verification results. Information collected at runtime can further update usage profiles and coverage regions, making Qu ac incremental and capable of providing more accurate modeling and risk assessment as information accumulates. This allows developers to combine static verification methods with runtime verification and testing methods to rigorously assess software reliability.