Verification of MPI programs

Andrew M. Mironov

DOI: https://doi.org/10.48550/arXiv.2110.08537

2021-10-16

Abstract:In this paper, we outline an approach to verifying parallel programs. A new mathematical model of parallel programs is introduced. The introduced model is illustrated by the verification of the matrix multiplication MPI program.

Logic in Computer Science,Programming Languages

What problem does this paper attempt to address?

Towards deductive verification of MPI programs against session types

Eduardo R. B. Marques,Francisco Martins,Vasco T. Vasconcelos,Nicholas Ng,Nuno Martins

DOI: https://doi.org/10.4204/EPTCS.137.9

2013-12-10

Abstract:The Message Passing Interface (MPI) is the de facto standard message-passing infrastructure for developing parallel applications. Two decades after the first version of the library specification, MPI-based applications are nowadays routinely deployed on super and cluster computers. These applications, written in C or Fortran, exhibit intricate message passing behaviours, making it hard to statically verify important properties such as the absence of deadlocks. Our work builds on session types, a theory for describing protocols that provides for correct-by-construction guarantees in this regard. We annotate MPI primitives and C code with session type contracts, written in the language of a software verifier for C. Annotated code is then checked for correctness with the software verifier. We present preliminary results and discuss the challenges that lie ahead for verifying realistic MPI program compliance against session types.

Distributed, Parallel, and Cluster Computing,Logic in Computer Science,Programming Languages
Matrix Multiplication Verification Using Coding Theory

Huck Bennett,Karthik Gajulapalli,Alexander Golovnev,Evelyn Warton

2024-07-20

Abstract:We study the Matrix Multiplication Verification Problem (MMV) where the goal is, given three $n \times n$ matrices $A$, $B$, and $C$ as input, to decide whether $AB = C$. A classic randomized algorithm by Freivalds (MFCS, 1979) solves MMV in $\widetilde{O}(n^2)$ time, and a longstanding challenge is to (partially) derandomize it while still running in faster than matrix multiplication time (i.e., in $o(n^{\omega})$ time). To that end, we give two algorithms for MMV in the case where $AB - C$ is sparse. Specifically, when $AB - C$ has at most $O(n^{\delta})$ non-zero entries for a constant $0 \leq \delta < 2$, we give (1) a deterministic $O(n^{\omega - \varepsilon})$-time algorithm for constant $\varepsilon = \varepsilon(\delta) > 0$, and (2) a randomized $\widetilde{O}(n^2)$-time algorithm using $\delta/2 \cdot \log_2 n + O(1)$ random bits. The former algorithm is faster than the deterministic algorithm of Künnemann (ESA, 2018) when $\delta \geq 1.056$, and the latter algorithm uses fewer random bits than the algorithm of Kimbrel and Sinha (IPL, 1993), which runs in the same time and uses $\log_2 n + O(1)$ random bits (in turn fewer than Freivalds's algorithm). We additionally study the complexity of MMV. We first show that all algorithms in a natural class of deterministic linear algebraic algorithms for MMV (including ours) require $\Omega(n^{\omega})$ time. We also show a barrier to proving a super-quadratic running time lower bound for matrix multiplication (and hence MMV) under the Strong Exponential Time Hypothesis (SETH). Finally, we study relationships between natural variants and special cases of MMV (with respect to deterministic $\widetilde{O}(n^2)$-time reductions).

Data Structures and Algorithms
Methods to Model-Check Parallel Systems Software

Olga Shumsky Matlin,William McCune,Ewing Lusk

DOI: https://doi.org/10.48550/arXiv.cs/0312012

2003-12-06

Abstract:We report on an effort to develop methodologies for formal verification of parts of the Multi-Purpose Daemon (MPD) parallel process management system. MPD is a distributed collection of communicating processes. While the individual components of the collection execute simple algorithms, their interaction leads to unexpected errors that are difficult to uncover by conventional means. Two verification approaches are discussed here: the standard model checking approach using the software model checker SPIN and the nonstandard use of a general-purpose first-order resolution-style theorem prover OTTER to conduct the traditional state space exploration. We compare modeling methodology and analyze performance and scalability of the two methods with respect to verification of MPD.

Logic in Computer Science,Distributed, Parallel, and Cluster Computing
Verifying Parallel Low-Level Programs for Multi-core Processor

Yun-min ZHU,Li-wei ZHANG,Sheng-yuan WANG,Yuan DONG,Su-qin ZHANG

DOI: https://doi.org/10.3321/j.issn:0372-2112.2009.z1.001

2009-01-01

Tien Tzu Hsueh Pao/Acta Electronica Sinica

Abstract:2 Abstract As the multi-core processor is widely used and advanced high-trusted software is required, the verification of parallel programs for multi-core processor becomes more and more important. This paper presents a proof framework about the verification of parallel programs, including the definition of our abstract machine, the formal specification for object code, logic inference rules and the proof of soundness theory. We certify the code at an assembly-level directly in order to disregard the correctness of compilers. The classic spin-lock technology is introduced to implement the mutually exclusive access to shared memory. Our proof framework supports Hoare-logic style reasoning. In addition, we use high-order logic to describe both operational semantics and security-policy, so the partial correctness of multi-core parallel programs can be verified in our system.
Distributed Bounded Model Checking

Prantik Chatterjee,Subhajit Roy,Bui Phi Diep,Akash Lal

DOI: https://doi.org/10.48550/arXiv.2005.08063

2020-05-17

Abstract:Program verification is a resource-hungry task. This paper looks at the problem of parallelizing SMT-based automated program verification, specifically bounded model-checking, so that it can be distributed and executed on a cluster of machines. We present an algorithm that dynamically unfolds the call graph of the program and frequently splits it to create sub-tasks that can be solved in parallel. The algorithm is adaptive, controlling the splitting rate according to available resources, and also leverages information from the SMT solver to split where most complexity lies in the search. We implemented our algorithm by modifying CORRAL, the verifier used by Microsoft's Static Driver Verifier (SDV), and evaluate it on a series of hard SDV benchmarks.

Programming Languages,Software Engineering
Parallel Software Model Checking

邝宏斌,罗贵明

DOI: https://doi.org/10.3969/j.issn.1000-3428.2008.19.009

2008-01-01

Abstract:Parallelism is an effective method in improving the efficiency of model checking. C program model checking based on LTS is studied and a methodology for parallel software model checking is presented. It utilizes modularity of MAGIC, a software model checker, to decompose C program into components, distributes them over computational nodes and parallel call MAGIC to complete the verification. Close to linear speedup and good scalability can be achieved due to little synchronization and inter-nodes communication. The reduction of time expense is beneficial to formal verification of large-scale software.
Parallel approach of three-dimensional phase-field model of binary alloy in MPI+OpenMP environment

Chang-sheng ZHU,Xin DENG,Li FENG,Hao LI

DOI: https://doi.org/10.3969/j.issn.1673-5196.2017.04.005

2017-01-01

Abstract:Aimed at the problems in simulative investigation of three-dimensional phase-field model of binary alloy such as small computation region, low computation efficiency, so that it is difficult to meet the requirements of large-scale simulation of binary alloy, a parallel method based on MPI+OpenMP hybrid programming mode is introduced.The advantages and disadvantages of both OpenMP and MPI parallel techniques are analyzed under conditions of identical calculation scope.The parallel efficiency of MPI+OpenMP hybrid programming mode and pure MPI parallel method is analyzed through comparison with each other in computer-cluster environment.It is verified by the result obtained that the calculation scope of three-dimensional phase-field model of binary alloy could be expanded effectively by using MPI+OpenMP hybrid programming mode.In multinuclear computer- cluster environment, The MPI +OpenMP hybrid programming mode could more effectively take the advantage of the multi-level structure of the multinuclear computer than the pure MPI parallel method and obtain an even higher speed ratio and computation efficiency.
A New Method of Verification of Functional Programs

Andrew M. Mironov

DOI: https://doi.org/10.48550/arXiv.1604.04240

2016-04-15

Abstract:In the paper the problem of verification of functional programs (FPs) over strings is considered, where specifications of properties of FPs are defined by other FPs, and a FP S1 meets a specification defined by another FP S2 iff a composition of functions defined by the FPs S1 and S2 is equal to the constant 1. We introduce a concept of a state diagram of a FP, and reduce the verification problem to the problem of an analysis of the state diagrams of FPs. The proposed approach is illustrated by the example of verification of a sorting program.

Logic in Computer Science
Verifying Programs Using Abstraction and Theorem Proving.

Junyan Qian,Baowen Xu

2007-01-01

Abstract:we present a methodology for automatically verifying programs against safety specifications based on finite state machine. Firstly, computing the abstract transition between abstract states is done by calling a theorem prover, and then an initial abstract model automatically is extracted from concrete program using predicate abstraction. However, the process of abstraction can be exponential in the number of predicates used. Program slicing, predicate inference and partitioning the set of candidate predicates into subsets make abstract model construction effective. By counterexample-guided abstraction refinement scheme, the abstraction refines incrementally until the specification is either satisfied or refuted. Finally, our methods can be extended to verifying concurrency programs by parallel composition.
To parallelize or not to parallelize, bugs issue

Alaa I. El-Nashar,Nakamura Masaki

DOI: https://doi.org/10.48550/arXiv.1311.0728

2013-11-04

Abstract:Program correctness is one of the most difficult challenges in parallel programming. Message Passing Interface MPI is widely used in writing parallel applications. Since MPI is not a compiled language, the programmer will be enfaced with several programming <a class="link-external link-http" href="http://bugs.This" rel="external noopener nofollow">this http URL</a> paper presents the most common programming bugs arise in MPI programs to help the programmer to compromise between the advantage of parallelism and the extra effort needed to detect and fix such bugs. An algebraic specification of an MPI-like programming language, called Simple MPI (SMPI), to be used in writing MPI programs specification has also been proposed. In addition, both nondeterminacy and deadlocks arise in SMPI programs have been verified using Maud system.

Programming Languages,Distributed, Parallel, and Cluster Computing
A graph model of message passing processes

Andrew M. Mironov

DOI: https://doi.org/10.48550/arXiv.1706.00266

2017-06-01

Abstract:In the paper we consider a graph model of message passing processes and present a method verification of message passing processes. The method is illustrated by an example of a verification of sliding window protocol.

Logic in Computer Science,Distributed, Parallel, and Cluster Computing
GPU-Accelerated Verification of Machine Learning Models for Power Systems

Samuel Chevalier,Ilgiz Murzakhanov,Spyros Chatzivasileiadis

2023-09-08

Abstract:Computational tools for rigorously verifying the performance of large-scale machine learning (ML) models have progressed significantly in recent years. The most successful solvers employ highly specialized, GPU-accelerated branch and bound routines. Such tools are crucial for the successful deployment of machine learning applications in safety-critical systems, such as power systems. Despite their successes, however, barriers prevent out-of-the-box application of these routines to power system problems. This paper addresses this issue in two key ways. First, for the first time to our knowledge, we enable the simultaneous verification of multiple verification problems (e.g., checking for the violation of all line flow constraints simultaneously and not by solving individual verification problems). For that, we introduce an exact transformation that converts the "worst-case" violation across a set of potential violations to a series of ReLU-based layers that augment the original neural network. This allows verifiers to interpret them directly. Second, power system ML models often must be verified to satisfy power flow constraints. We propose a dualization procedure which encodes linear equality and inequality constraints directly into the verification problem; and in a manner which is mathematically consistent with the specialized verification tools. To demonstrate these innovations, we verify problems associated with data-driven security constrained DC-OPF solvers. We build and test our first set of innovations using the $\alpha,\beta$-CROWN solver, and we benchmark against Gurobi 10.0. Our contributions achieve a speedup that can exceed 100x and allow higher degrees of verification flexibility.

Machine Learning,Systems and Control
A Novel Approach for Bounded Model Checking Through Full Parallelism

Debao Sang,Jing Liu,Haiying Sun,Jin Xu,Jiexiang Kang

DOI: https://doi.org/10.1109/qrs57517.2022.00046

2022-01-01

Abstract:Bounded Model Checking (BMC) has been found promising in finding deep vulnerabilities in industry designs and scaling well with design sizes. However, the parallelisation of BMC is challenging, due to the propositional satisfiability (SAT) problem and satisfiability modulo theories problem solving being hard to parallelise. In this paper, we propose a novel approach to perform BMC based on the mathematical model of probe machine, which is the first approach to employ probe machine to accelerate BMC, particularly it can solve SAT formulas in full parallel. We introduce the workflow of the algorithm and explain in detail the process of mapping BMC to the probe machine. A method is provided to prove the correctness of the algorithm and to analyze its time complexity. We develop a model checker called BMC2PROBE based on our approach and explain the framework and memory management of the tool. The experiment results are discussed, which prove the feasibility and effectiveness of our approach.
Verification of Concrete Programs with Respect to Abstract Programs

Bin LI,Zhen-Hao TANG,Juan ZHAI,Jian-Hua ZHAO

DOI: https://doi.org/10.13328/j.cnki.jos.005195

2017-01-01

Abstract:This paper presents an approach to prove that a concrete program correctly implements its corresponding abstract program.Here,an abstract program uses some abstract data types such as set,list and map,and abstract operations upon those data types.A concrete program uses the types in the C-like language.The approach presented in the paper requires to specify correspondences between the abstract program and the concrete program,including correspondences between program points and correspondences between variables.Based on the correspondences,the verification task can be divided into small subtasks that can be easily and mostly automatically verified.
MPIrigen: MPI Code Generation through Domain-Specific Language Models

Nadav Schneider,Niranjan Hasabnis,Vy A. Vo,Tal Kadosh,Neva Krien,Mihai Capotă,Guy Tamir,Ted Willke,Nesreen Ahmed,Yuval Pinter,Timothy Mattson,Gal Oren

2024-04-24

Abstract:The imperative need to scale computation across numerous nodes highlights the significance of efficient parallel computing, particularly in the realm of Message Passing Interface (MPI) integration. The challenging parallel programming task of generating MPI-based parallel programs has remained unexplored. This study first investigates the performance of state-of-the-art language models in generating MPI-based parallel programs. Findings reveal that widely used models such as GPT-3.5 and PolyCoder (specialized multi-lingual code models) exhibit notable performance degradation, when generating MPI-based programs compared to general-purpose programs. In contrast, domain-specific models such as MonoCoder, which are pretrained on MPI-related programming languages of C and C++, outperform larger models. Subsequently, we introduce a dedicated downstream task of MPI-based program generation by fine-tuning MonoCoder on HPCorpusMPI. We call the resulting model as MPIrigen. We propose an innovative preprocessing for completion only after observing the whole code, thus enabling better completion with a wider context. Comparative analysis against GPT-3.5 zero-shot performance, using a novel HPC-oriented evaluation method, demonstrates that MPIrigen excels in generating accurate MPI functions up to 0.8 accuracy in location and function predictions, and with more than 0.9 accuracy for argument predictions. The success of this tailored solution underscores the importance of domain-specific fine-tuning in optimizing language models for parallel computing code generation, paving the way for a new generation of automatic parallelization tools. The sources of this work are available at our GitHub MPIrigen repository:

Distributed, Parallel, and Cluster Computing,Artificial Intelligence,Computation and Language,Machine Learning,Software Engineering
To parallelize or not to parallelize, control and data flow issue

Alaa I. Elnashar

DOI: https://doi.org/10.48550/arXiv.1311.0731

2013-11-04

Abstract:New trends towards multiple core processors imply using standard programming models to develop efficient, reliable and portable programs for distributed memory multiprocessors and workstation PC clusters. Message passing using MPI is widely used to write efficient, reliable and portable applications. Control and data flow analysis concepts, techniques and tools are needed to understand and analyze MPI programs. If our point of interest is the program control and data flow analysis, to decide to parallelize or not to parallelize our applications, there is a question to be answered, " Can the existing concepts, techniques and tools used to analyze sequential programs also be used to analyze parallel ones written in MPI?". In this paper we'll try to answer this question.

Programming Languages,Distributed, Parallel, and Cluster Computing
A Formal Approach for Modeling and Verification of Distributed Systems.

Gang Ren,Pan Deng,Chao Yang,Jianwei Zhang,Qingsong Hua

DOI: https://doi.org/10.1007/978-3-319-38904-2_33

2015-01-01

Abstract:In recent year, distributed systems have become a mainstream paradigm in industry and how to ensure correctness and reliability is a great challenge for practicing engineers. Therefore, in this paper a formal approach is proposed for modelling and verification of distributed systems, which integrates UML sequence diagram, pi-calculus and NuSMV within one framework. Moreover, the practicality of the proposed approach is illuminated though a case study of scheduling road emergency service.
Verification of deep probabilistic models

Krishnamurthy Dvijotham,Marta Garnelo,Alhussein Fawzi,Pushmeet Kohli

DOI: https://doi.org/10.48550/arXiv.1812.02795

2018-12-07

Abstract:Probabilistic models are a critical part of the modern deep learning toolbox - ranging from generative models (VAEs, GANs), sequence to sequence models used in machine translation and speech processing to models over functional spaces (conditional neural processes, neural processes). Given the size and complexity of these models, safely deploying them in applications requires the development of tools to analyze their behavior rigorously and provide some guarantees that these models are consistent with a list of desirable properties or specifications. For example, a machine translation model should produce semantically equivalent outputs for innocuous changes in the input to the model. A functional regression model that is learning a distribution over monotonic functions should predict a larger value at a larger input. Verification of these properties requires a new framework that goes beyond notions of verification studied in deterministic feedforward networks, since requiring worst-case guarantees in probabilistic models is likely to produce conservative or vacuous results. We propose a novel formulation of verification for deep probabilistic models that take in conditioning inputs and sample latent variables in the course of producing an output: We require that the output of the model satisfies a linear constraint with high probability over the sampling of latent variables and for every choice of conditioning input to the model. We show that rigorous lower bounds on the probability that the constraint is satisfied can be obtained efficiently. Experiments with neural processes show that several properties of interest while modeling functional spaces can be modeled within this framework (monotonicity, convexity) and verified efficiently using our algorithms

Machine Learning
Code Generation Verification – Assessing Numerical Equivalence between Simulink Models and Generated Code

G. Sandmann,M. Conrad,T. Erkkinen,Thomas Maier-Komor,Marty Pomeroy

Abstract:With the advent of Model-Based Design and production code generation, the ability to demonstrate that the generated object code correctly implements the model used for code generation, i.e. to show the correctness of the model-to-code transformation result, becomes inherent. Various approaches to verify the model-to-code translation result in practice, i.e., to efficiently and effectively assess the correctness of translating real word Simulink models into embedded C code, have been proposed and applied by engineering professionals. Unfortunately, many of these approaches do not exploit the advantages of Model-Based Design and more or less carry out the translation validation of generated code in the same manner as for hand-written code. In this paper, the authors will present Code Generation Verification (CGV) an automated, testing-based approach to assess the numerical equivalence between Simulink models and the generated code. In the CGV approach, each individual model-tocode translation is followed by a verification phase to assess that the input (i.e., the Simulink model used for code generation) and the output (i.e., the target code produced during code generation and compilation) of this translation produce the same numerical results when stimulated with identical inputs. The paper describes the proposed verification approach, and illustrates it by using Simulink and Real-Time Workshop Embedded Coder. It also discusses this approach in the context of high-integrity application development by embedding it into code verification workflows for IEC 61508 and ISO 26262.

Engineering,Computer Science
Certifying MIP-based Presolve Reductions for 0-1 Integer Linear Programs

Alexander Hoen,Andy Oertel,Ambros Gleixner,Jakob Nordström

2024-03-20

Abstract:It is well known that reformulating the original problem can be crucial for the performance of mixed-integer programming (MIP) solvers. To ensure correctness, all transformations must preserve the fea sibility status and optimal value of the problem, but there is currently no established methodology to express and verify the equivalence of two mixed-integer programs. In this work, we take a first step in this direction by showing how the correctness of MIP presolve reductions on 0-1 integer linear programs can be certified by using (and suitably extending) the VeriPB tool for pseudo-Boolean proof logging. Our experimental evaluation on both decision and optimization instances demonstrates the computational viability of the approach and leads to suggestions for future revisions of the proof format that will help to reduce the verbosity of the certificates and to accelerate the certification and verification process further.

Optimization and Control

Verification of MPI programs

Towards deductive verification of MPI programs against session types

Matrix Multiplication Verification Using Coding Theory

Methods to Model-Check Parallel Systems Software

Verifying Parallel Low-Level Programs for Multi-core Processor

Distributed Bounded Model Checking

Parallel Software Model Checking

Parallel approach of three-dimensional phase-field model of binary alloy in MPI+OpenMP environment

A New Method of Verification of Functional Programs

Verifying Programs Using Abstraction and Theorem Proving.

To parallelize or not to parallelize, bugs issue

A graph model of message passing processes

GPU-Accelerated Verification of Machine Learning Models for Power Systems

A Novel Approach for Bounded Model Checking Through Full Parallelism

Verification of Concrete Programs with Respect to Abstract Programs

MPIrigen: MPI Code Generation through Domain-Specific Language Models

To parallelize or not to parallelize, control and data flow issue

A Formal Approach for Modeling and Verification of Distributed Systems.

Verification of deep probabilistic models

Code Generation Verification – Assessing Numerical Equivalence between Simulink Models and Generated Code

Certifying MIP-based Presolve Reductions for 0-1 Integer Linear Programs