ParFORM: recent development

M.Tentyukov,J.A.M.Vermaseren,H.M.Staudenmaier

DOI: https://doi.org/10.1016/j.nima.2005.11.142

2005-10-31

Abstract:We report on the status of our project of parallelization of the symbolic manipulation program FORM. We have now parallel versions of FORM running on Cluster- or SMP-architectures. These versions can be used to run arbitrary FORM programs in parallel.

Symbolic Computation

What problem does this paper attempt to address?

ParFORM: Parallel Version of the Symbolic Manipulation Program FORM

M.Tentyukov,D.Fliegner,M.Frank,A.Onischenko,A.Retey,H.M.Staudenmaier,J.A.M.Vermaseren

DOI: https://doi.org/10.48550/arXiv.cs/0407066

2004-07-30

Abstract:After an introduction to the sequential version of FORM and the mechanisms behind, we report on the status of our project of parallelization. We have now a parallel version of FORM running on Cluster- and SMP-architectures. This version can be used to run arbitrary FORM programs in parallel.

Symbolic Computation,Distributed, Parallel, and Cluster Computing,High Energy Physics - Phenomenology
Parallelized Implementation of the Finite Particle Method for Explicit Dynamics in GPU

Jingzhe Tang,Yanfeng Zheng,Chao Yang,Wei Wang,Yaozhi Luo

DOI: https://doi.org/10.32604/cmes.2020.08104

2020-01-01

Computer Modeling in Engineering & Sciences

Abstract:As a novel kind of particle method for explicit dynamics, the finite particle method (FPM) does not require the formation or solution of global matrices, and the evaluations of the element equivalent forces and particle displacements are decoupled in nature, thus making this method suitable for parallelization. The FPM also requires an acceleration strategy to overcome the heavy computational burden of its explicit framework for time-dependent dynamic analysis. To this end, a GPU-accelerated parallel strategy for the FPM is proposed in this paper. By taking advantage of the independence of each step of the FPM workflow, a generic parallelized computational framework for multiple types of analysis is established. Using the Compute Unified Device Architecture (CUDA), the GPU implementations of the main tasks of the FPM, such as evaluating and assembling the element equivalent forces and solving the kinematic equations for particles, are elaborated through careful thread management and memory optimization. Performance tests show that speedup ratios of 8, 25 and 48 are achieved for beams, hexahedral solids and triangular shells, respectively. For examples consisting of explicit dynamic analyses of shells and solids, comparisons with Abaqus using 1 to 8 CPU cores validate the accuracy of the results and demonstrate a maximum speed improvement of a factor of 11.2.
A Parallel Simulator for Massive Reservoir Models Utilizing Distributed-Memory Parallel Systems

Hui Liu,Lihua Shen,Yan Chen,Kun Wang,Bo Yang,Zhangxin Chen

DOI: https://doi.org/10.48550/arXiv.1701.06254

2017-01-23

Abstract:This paper presents our work on developing parallel computational methods for two-phase flow on modern parallel computers, where techniques for linear solvers and nonlinear methods are studied and the standard and inexact Newton methods are investigated. A multi-stage preconditioner for two-phase flow is applied and advanced matrix processing strategies are studied. A local reordering method is developed to speed the solution of linear systems. Numerical experiments show that these computational methods are effective and scalable, and are capable of computing large-scale reservoir simulation problems using thousands of CPU cores on parallel computers. The nonlinear techniques, preconditioner and matrix processing strategies can also be applied to three-phase black oil, compositional and thermal models.

Computational Engineering, Finance, and Science
FORM version 4.2

Ben Ruijl,Takahiro Ueda,Jos Vermaseren

DOI: https://doi.org/10.48550/arXiv.1707.06453

2017-07-20

Abstract:We introduce FORM 4.2, a new minor release of the symbolic manipulation toolkit. We demonstrate several new features, such as a new pattern matching option, new output optimization, and automatic expansion of rational functions.

High Energy Physics - Phenomenology,Symbolic Computation
PARSIR: a Package for Effective Parallel Discrete Event Simulation on Multi-processor Machines

Francesco Quaglia

2024-10-01

Abstract:In this article we present PARSIR (PARallel SImulation Runner), a package that enables the effective exploitation of shared-memory multi-processor machines for running discrete event simulation models. PARSIR is a compile/run-time environment for discrete event simulation models developed with the {\tt C} programming language. The architecture of PARSIR has been designed in order to keep low the amount of CPU-cycles required for running models. This is achieved via the combination of a set of techniques like: 1) causally consistent batch-processing of simulation events at an individual simulation object for caching effectiveness; 2) high likelihood of disjoint access parallelism; 3) the favoring of memory accesses on local NUMA (Non-Uniform-Memory-Access) nodes in the architecture, while still enabling well balanced workload distribution via work-stealing from remote nodes; 4) the use of RMW (Read-Modify-Write) machine instructions for fast access to simulation engine data required by the worker threads for managing the concurrent simulation objects and distributing the workload. Furthermore, any architectural solution embedded in the PARSIR engine is fully transparent to the application level code implementing the simulation model. We also provide experimental results showing the effectiveness of PARSIR when running the reference PHOLD benchmark on a NUMA shared-memory multi-processor machine equipped with 40 CPUs.

Distributed, Parallel, and Cluster Computing
Parallelization in Calculational Forms.

Zhenjiang Hu,Masato Takeichi,Wei-Ngan Chin

DOI: https://doi.org/10.1145/268946.268972

1998-01-01

Abstract:The problems involved in developing efficient parallel programs have proved harder than those in developing efficient sequential ones, both for programmers and for compilers. Although program calculation has been found to be a promising way to solve these problems in the sequential world, we believe that it needs much more effort to study its effective use in the parallel world. In this paper, we propose a calculational framework for the derivation of efficient parallel programs with two main innovations:. -We propose a novel inductive synthesis lemma based on which an elementary but powerful parallelization theorem is developed. -We make the first attempt to construct a calculational algorithm for parallelization, deriving associative operators from data type definition and making full use of existing fusion and tupling calculations.Being more constructive, our method is not only helpful in the design of efficient parallel programs in general but also promising in the construction of parallelizing compiler. Several interesting examples are used for illustration.
OMPar: Automatic Parallelization with AI-Driven Source-to-Source Compilation

Tal Kadosh,Niranjan Hasabnis,Prema Soundararajan,Vy A. Vo,Mihai Capota,Nesreen Ahmed,Yuval Pinter,Gal Oren

2024-09-23

Abstract:Manual parallelization of code remains a significant challenge due to the complexities of modern software systems and the widespread adoption of multi-core architectures. This paper introduces OMPar, an AI-driven tool designed to automate the parallelization of C/C++ code using OpenMP pragmas. OMPar integrates Large Language Models (LLMs) through two key components: OMPify, which assesses loop parallelization potential, and MonoCoder-OMP, a new fine-tuned model which generates precise OpenMP pragmas. The evaluation of OMPar follows the same rigorous process applied to traditional tools like source-to-source AutoPar and ICPC compilers: (1) ensuring the generated code compiles and runs correctly in serial form, (2) assessing performance with the gradual addition of threads and corresponding physical cores, and (3) verifying and validating the correctness of the code's output. Benchmarks from HeCBench and ParEval are used to evaluate accuracy and performance. Experimental results demonstrate that OMPar significantly outperforms traditional methods, achieving higher accuracy in identifying parallelizable loops and generating efficient pragmas. Beyond accuracy, OMPar offers advantages such as the ability to work on partial or incomplete codebases and the capacity to continuously learn from new code patterns, enhancing its parallelization capabilities over time. These results underscore the potential of LLMs in revolutionizing automatic parallelization techniques, paving the way for more efficient and scalable parallel computing systems.

Computation and Language
Parallelizing Prolog on Shared-Memory Multiprocessors

Gao Yaoqing,Wang Dingxing,Qiu Xiaolin,Zhiyi Hwang,Shouren Hu

DOI: https://doi.org/10.1007/BFb0013541

1991-01-01

Abstract:The research goal is to design and evaluate a parallel computer system to execute logic programs in both AND-parallel and OR-parallel. In this paper, we first introduce the OR-forest description. Then we provide a brief overview of the OR-forest-based parallel model RAP/LOP and its parallel abstract machine. We focus on the issues relevant to the parallelization of Prolog at compile-time. The simulation system of the RAP/LOP-PIM has been implemented on the SUN workstation and VAX 11/780.
Parallel Computing of Shared Memory Multiprocessors Based on JOMP

Zhang Hong,Cao Jie,Wang Xiaoming,Zhu Changsheng

DOI: https://doi.org/10.2991/meic-15.2015.346

2015-01-01

Abstract:Aiming at parallel computing of shared memory multiprocessors based on JOMP, this paper probes the writing, compiling and running of a (star). jomp file in detail and describes a complete specification and process for parallel programming. It expounds the format of parallel directive sets, main runtime libraries and their functions, making a further illustration of variable attributes in parallel region. Taking an example of computing values of pi on a lenovo Intel(R) Core(TM) i3-2120, it obtained a 1.73 speedup factor and 86.5% efficiency, and proved the feasibility and effectiveness of this system.
Parthenon—a performance portable block-structured adaptive mesh refinement framework

Philipp Grete,Joshua C Dolence,Jonah M Miller,Joshua Brown,Ben Ryan,Andrew Gaspar,Forrest Glines,Sriram Swaminarayan,Jonas Lippuner,Clell J Solomon,Galen Shipman,Christoph Junghans,Daniel Holladay,James M Stone,Luke F Roberts

DOI: https://doi.org/10.1177/10943420221143775

2022-12-15

Abstract:The International Journal of High Performance Computing Applications, Ahead of Print. On the path to exascale the landscape of computer device architectures and corresponding programming models has become much more diverse. While various low-level performance portable programming models are available, support at the application level lacks behind. To address this issue, we present the performance portable block-structured adaptive mesh refinement (AMR) framework Parthenon, derived from the well-tested and widely used Athena++ astrophysical magnetohydrodynamics code, but generalized to serve as the foundation for a variety of downstream multi-physics codes. Parthenon adopts the Kokkos programming model, and provides various levels of abstractions from multidimensional variables, to packages defining and separating components, to launching of parallel compute kernels. Parthenon allocates all data in device memory to reduce data movement, supports the logical packing of variables and mesh blocks to reduce kernel launch overhead, and employs one-sided, asynchronous MPI calls to reduce communication overhead in multi-node simulations. Using a hydrodynamics miniapp, we demonstrate weak and strong scaling on various architectures including AMD and NVIDIA GPUs, Intel and AMD x86 CPUs, IBM Power9 CPUs, as well as Fujitsu A64FX CPUs. At the largest scale on Frontier (the first TOP500 exascale machine), the miniapp reaches a total of 1.7 × 1013 zone-cycles/s on 9216 nodes (73,728 logical GPUs) at [math] weak scaling parallel efficiency (starting from a single node). In combination with being an open, collaborative project, this makes Parthenon an ideal framework to target exascale simulations in which the downstream developers can focus on their specific application rather than on the complexity of handling massively-parallel, device-accelerated AMR.

computer science, theory & methods, interdisciplinary applications, hardware & architecture
Optimized Parallel Execution of Declarative Programs on Distributed Memory Multiprocessors

Meiming Shen,Xinmin Tian,Dingxing Wang,Weimin Zheng,Dongchan Wen

DOI: https://doi.org/10.1007/bf02939530

1993-01-01

Abstract:In this paper, we focus on the compiling implementation of parallel logic language PARLOG and functional language ML on distributed memory multiprocessors. Under the graph rewriting framework, a Heterogeneous Parallel Graph Rewriting Execution Model (HPGREM) is presented firstly. Then based on HPGREM, a parallel abstract machine PAM/TGR is described. Furthermore, several optimizing compilation schemes for executing declarative programs on transputer array are proposed. The performance statistics on a transputer array demonstrate the effectiveness of our model, parallel abstract machine, optimizing compilation strategies and compiler.
J-Parallelio -- automatic parallelization framework for Java virtual machine code

Krzysztof Stuglik,Piotr Listkiewicz,Mateusz Kulczyk,Marcin Pietron

DOI: https://doi.org/10.48550/arXiv.2303.08746

2023-02-07

Distributed, Parallel, and Cluster Computing

Abstract:Manual translation of the algorithms from sequential version to its parallel counterpart is time consuming and can be done only with the specific knowledge of hardware accelerator architecture, parallel programming or programming environment. The automation of this process makes porting the code much easier and faster. The key aspect in this case is how efficient the generated parallel code will be. The paper describes J-Parallelio, the framework for automatic analysis of the bytecode source codes and its parallelisation on multicore processors. The process consists of a few steps. First step is a process of decompilation of JVM and its translation to internal abstract syntax tree, the dependency extraction and memory analysis is performed. Finally, the mapping process is performed which consists of a set of rules responsible for translating the input virtual machine source code to its parallel version. The main novelty is that it can deal with pure Java virtual machine and can generate parallel code for multicore processors. This makes the system portable and it can work with different languages based on JVM after some small modifications. The efficiency of automatically translated source codes were compared with their manually written counterparts on chosen benchmarks.
ParaInsight: an Assistant for Quantitatively Analyzing Multi-granularity Parallel Region.

Ran Ao,Guangming Tan,Mingyu Chen

DOI: https://doi.org/10.1109/hpcc.and.euc.2013.103

2013-01-01

Abstract:With the growing number of cores on chip, scalability of application becomes one of the most important metrics to impact performance and efficient utilization of hardware resource. Simple multi-threading transformation usually acts not well enough so that many of multi-threaded programs should be reconsidered for exploiting more parallelism. However, complex data dependency makes programmer hard to discover the regions with good scalability. Meanwhile, a variety of parallel programming models and platform properties make the choice of parallel granularity even difficult. In this paper, we develop a tool, named Para Insight, to help programmer obtain a quantification insight of parallelism on multi-granularity, as an assistant to ease the process of parallel analysis. We concentrate on revealing the inherent characteristic of program itself, independent of programming model and platform. While regions on which parallelization efforts should be focused are ordering, programmer could choose interested candidate and then use the quantification results to estimate the parallel zing overhead according to concrete parallel programming model and platform parameters. In contrast to many other parallelism discovery tools which mainly focus on parallel zing legacy serial programs, our tool provides assistance on the process of multi-threaded program development. We use multi-threaded benchmark suite Parsec and Splash2 to demonstrate the use of our tool, and specially use two cases to present how to improve performance of multithreaded application with the utilization of analysis data.
PARRAY

Yifeng Chen,Xiang Cui,Hong Mei

DOI: https://doi.org/10.1145/2145816.2145838

2012-01-01

Abstract:This paper introduces a programming interface called PARRAY (or Parallelizing ARRAYs) that supports system-level succinct programming for heterogeneous parallel systems like GPU clusters. The current practice of software development requires combining several low-level libraries like Pthread, OpenMP, CUDA and MPI. Achieving productivity and portability is hard with different numbers and models of GPUs. PARRAY extends mainstream C programming with novel array types of distinct features: 1) the dimensions of an array type are nested in a tree, conceptually reflecting the memory hierarchy; 2) the definition of an array type may contain references to other array types, allowing sophisticated array types to be created for parallelization; 3) threads also form arrays that allow programming in a Single-Program-Multiple-Codeblock (SPMC) style to unify various sophisticated communication patterns. This leads to shorter, more portable and maintainable parallel codes, while the programmer still has control over performance-related features necessary for deep manual optimization. Although the source-to-source code generator only faithfully generates low-level library calls according to the type information, higher-level programming and automatic performance optimization are still possible through building libraries of sub-programs on top of PARRAY. The case study on cluster FFT illustrates a simple 30-line code that 2x outperforms Intel Cluster MKL on the Tianhe-1A system with 7168 Fermi GPUs and 14336 CPUs.
Parallel Computation of Combinatorial Symmetries

Markus Anders,Pascal Schweitzer

DOI: https://doi.org/10.48550/arXiv.2108.04590

2021-08-10

Abstract:In practice symmetries of combinatorial structures are computed by transforming the structure into an annotated graph whose automorphisms correspond exactly to the desired symmetries. An automorphism solver is then employed to compute the automorphism group of the constructed graph. Such solvers have been developed for over 50 years, and highly efficient sequential, single core tools are available. However no competitive parallel tools are available for the task. We introduce a new parallel randomized algorithm that is based on a modification of the individualization-refinement paradigm used by sequential solvers. The use of randomization crucially enables parallelization. We report extensive benchmark results that show that our solver is competitive to state-of-the-art solvers on a single thread, while scaling remarkably well with the use of more threads. This results in order-of-magnitude improvements on many graph classes over state-of-the-art solvers. In fact, our tool is the first parallel graph automorphism tool that outperforms current sequential tools.

Data Structures and Algorithms
Parallel Algorithms for Dynamics Analysis of Structures and Their Impementations

Zhenhan Yao

1998-01-01

Abstract:The parallel algorithms for FEM in the distributed memory environment are studied. Two implicit parallel time integration algorithms are presented, which are, respectively, the paralleled Newmark method and Wilsion θ method. Emphasis is placed on the parallel LDL T decomposition of the equivalent stiffness matrices in the two methods. The algorithms are carried out in the MIMD environment on Transputer, and are also integrated into the available PFEM package which is a serial parallel mixed FEM softward. As verifying examples, a plane problem in elasticity and a 3D plate problem are presented. The result shows that the parallel efficiencies in present method are satisfactory as well, which are 0.70 and 0.55 respectively when the degree of freedom (DOF) is 7579, and the maximum bandwidth is 726 under two processors and three processors.
Making Formulog Fast: An Argument for Unconventional Datalog Evaluation (Extended Version)

Aaron Bembenek,Michael Greenberg,Stephen Chong

2024-09-26

Abstract:By combining Datalog, SMT solving, and functional programming, the language Formulog provides an appealing mix of features for implementing SMT-based static analyses (e.g., refinement type checking, symbolic execution) in a natural, declarative way. At the same time, the performance of its custom Datalog solver can be an impediment to using Formulog beyond prototyping -- a common problem for Datalog variants that aspire to solve large problem instances. In this work we speed up Formulog evaluation, with surprising results: while 2.2x speedups are obtained by using the conventional techniques for high-performance Datalog (e.g., compilation, specialized data structures), the big wins come by abandoning the central assumption in modern performant Datalog engines, semi-naive Datalog evaluation. In its place, we develop eager evaluation, a concurrent Datalog evaluation algorithm that explores the logical inference space via a depth-first traversal order. In practice, eager evaluation leads to an advantageous distribution of Formulog's SMT workload to external SMT solvers and improved SMT solving times: our eager evaluation extensions to the Formulog interpreter and Soufflé's code generator achieve mean 5.2x and 7.6x speedups, respectively, over the optimized code generated by off-the-shelf Soufflé on SMT-heavy Formulog benchmarks. Using compilation and eager evaluation, Formulog implementations of refinement type checking, bottom-up pointer analysis, and symbolic execution achieve speedups on 20 out of 23 benchmarks over previously published, hand-tuned analyses written in F#, Java, and C++, providing strong evidence that Formulog can be the basis of a realistic platform for SMT-based static analysis. Moreover, our experience adds nuance to the conventional wisdom that semi-naive evaluation is the one-size-fits-all best Datalog evaluation algorithm for static analysis workloads.

Programming Languages
Parallel Execution of Prolog on Shared-Memory Multiprocessors

Yaoqing Gao,Dingxing Wang,Weimin Zheng,Meiming Shen,Zhiyi Huang,Shouren Hu,Giorgio Levi

DOI: https://doi.org/10.1007/bf02939540

1993-01-01

Abstract:Logic programs offer many opportunities for the exploitation of parallelism. But the parallel execution of a task incurs various overheads. This paper focuses on the issues relevant to parallelizing Prolog on shared-memory multiprocessors efficiently.
FPM: A Flexible Programming Model for MPSoC on FPGA

Chao Wang,Xi Li,Junneng Zhang,Peng Chen,Xiaojing Feng,Xuehai Zhou

DOI: https://doi.org/10.1109/IPDPSW.2012.62

2012-01-01

Abstract:This paper proposes a flexible programming model (FPM), which addresses the automatic parallel execution for functional tasks on heterogeneous multiprocessors. Guided by the simply annotated source codes, a front-end source to source compiler is provided to identify the parallel regions and generate the sources codes. A runtime middleware analyzes the inter-task data dependencies and schedules the tasks with renaming techniques automatically. FPM has been verified by the prototype built on state-of-art FPGA. Examples demonstrate that our model can largely ease the burden of programmers as well as uncover the task level parallelism.
PaRSEC: Scalability, flexibility, and hybrid architecture support for task-based applications in ECP

Aurelien Bouteiller,Thomas Herault,Qinglei Cao,Joseph Schuchart,George Bosilca

DOI: https://doi.org/10.1177/10943420241290520

2024-10-19

The International Journal of High Performance Computing Applications

Abstract:The International Journal of High Performance Computing Applications, Ahead of Print. This paper highlights the most significant enhancements made to PaRSEC, a scalable task-based runtime system designed for hybrid machines, during the Exascale Computing Project (ECP). The enhancements focus on expanding the capabilities of PaRSEC to address the evolving landscape of parallel computing. Notable achievements include the integration of support for three major types of accelerators (NVIDIA, AMD, and Intel GPUs), the refinement and increased flexibility of the communication subsystem, and the introduction of new programming interfaces tailored for irregular applications. Additionally, the project resulted in the development of powerful debugging and performance analysis tools aimed at assisting users in understanding and optimizing their applications. We present a comprehensive demonstration of these advancements through a series of benchmarks and applications within ECP and beyond, thereby showcasing the enhanced capabilities of PaRSEC across the diverse architectures within the ECP, providing valuable insights into the runtime system's adaptability and performance across varied computing environments.

computer science, theory & methods, interdisciplinary applications, hardware & architecture

ParFORM: recent development

ParFORM: Parallel Version of the Symbolic Manipulation Program FORM

Parallelized Implementation of the Finite Particle Method for Explicit Dynamics in GPU

A Parallel Simulator for Massive Reservoir Models Utilizing Distributed-Memory Parallel Systems

FORM version 4.2

PARSIR: a Package for Effective Parallel Discrete Event Simulation on Multi-processor Machines

Parallelization in Calculational Forms.

OMPar: Automatic Parallelization with AI-Driven Source-to-Source Compilation

Parallelizing Prolog on Shared-Memory Multiprocessors

Parallel Computing of Shared Memory Multiprocessors Based on JOMP

Parthenon—a performance portable block-structured adaptive mesh refinement framework

Optimized Parallel Execution of Declarative Programs on Distributed Memory Multiprocessors

J-Parallelio -- automatic parallelization framework for Java virtual machine code

ParaInsight: an Assistant for Quantitatively Analyzing Multi-granularity Parallel Region.

PARRAY

Parallel Computation of Combinatorial Symmetries

Parallel Algorithms for Dynamics Analysis of Structures and Their Impementations

Making Formulog Fast: An Argument for Unconventional Datalog Evaluation (Extended Version)

Parallel Execution of Prolog on Shared-Memory Multiprocessors

FPM: A Flexible Programming Model for MPSoC on FPGA

PaRSEC: Scalability, flexibility, and hybrid architecture support for task-based applications in ECP