NUAPC: A Parallelizing Compiler for C++

Zhu Genjiang,Xie Li,Sun Zhongxiu

DOI: https://doi.org/10.1007/bf02943177

1997-01-01

Abstract:This paper presents a model for automatically parallelizing compiler based on C++ which consists of compile-time and run-time parallelizing facilities. The paper also describes a method for finding both intra-object and inter-object parallelism. The parallelism detection is completely transparent to users.

What problem does this paper attempt to address?

Design, Implementation of the Parallel C Language Based on C/S Mode in Distributed Systems

Xiaohui Zou

DOI: https://doi.org/10.1109/tmee.2011.6199258

2011-01-01

Abstract:In this paper, we designed Parallel C Language, a kind of parallel programming Language, to implement parallel computing in distributed systems. It is to add some special identification statements on ANSI C Language. The parallel computing is based on C/S mode, incorporating multithread and RPC(remote procedure call). Multithread is used to cut tasks and concurrent the subtasks, and RPC is used to deploy and execute the subtasks on remote server nodes. The allocation of remote server nodes is dependent on load balancing mechanism. We also implemented the Pre-compiler of this Parallel C Language program, which parses the Parallel C Language program into several RPC application files. The other RPC application files are constructed by RPCGEN. The result of test shows that using this Parallel C Language to program in distributed systems reduces the codes size and makes full use of the system resources and improves efficiency.
A New Methodology of Data Dependence Analysis for Parallelizing C++

GJ ZHU,L XIE,ZX SUN

DOI: https://doi.org/10.1145/219726.219744

1995-01-01

Abstract:This paper details parallelism detection in NUAPC[4], an automatically parallelizing compiler based on C++. Two kinds of parallelism, i.e. the inter- and intra-object parallelism, are introduced in NUAPC. The paper presents the method for seeking these parallelism based on the key structures Conflict Set and method Reference Vector, and proposes a new theoretical formation for expressing and performing data dependence analysis in object-oriented program paradigm. Data dependence involving inherited data and function members are also discussed.
A Path-Based Method of Parallelizing C++ Programs

GJ ZHU,L XIE,ZX SUN

DOI: https://doi.org/10.1145/181748.181751

1994-01-01

Abstract:This paper presents a path-based method for identifying parallelism in C++ programs and proposes an execution model which trades off the efficiency and parallelism. Based on the path information provided at compiler time or computed at run time, the model reduces the rollbacks while enhances the parallelism.
Hybrid Performance Modeling And Analyzing Of Parallel Systems

Bin Cheng,Weiqin Tong,Xingang Wang

2012-01-01

International Journal of Numerical Analysis and Modeling

Abstract:Performance is a key feature of parallel system. However, there is a great gap between the peak performance and performance attainable by a practical application. The model-based performance evaluation may be used to support the performance-oriented program development for parallel system. In this paper a hybrid TCPN model is proposed to describe the parallel program and the resources respectively. This method can bring less effect to modify the program structure because of running environment changes. And the performance engineering activities based on this model ranges from performance prediction in early development stages, performance analysis in the coding phase, to locate the performance bottleneck and modify it. After the correctness verification of the TCPN model, a reachable graph can be got. Then the further performance-tuning can be done by summing the execution time of corresponding action in the critical path.
Shared Variable Oriented Parallel Precompiler for SPMD Model.

Kang Jichang,Zhu Yi’an,Hong Yuanlin,Ying Bishan

DOI: https://doi.org/10.1007/bf02948344

1995-01-01

Abstract:For the moment, commercial parallel computer systems with distributed memory architecture are usually provided with parallel FORTRAN or parallel C compliers, which are just traditional sequential FORTRAN or C compilers expanded with communication statements. Programmers suffer from writing parallel programs with communication statements.
An Object Oriented C++ Parallel Compiler System

Xiao Nong,Shouren HU

DOI: https://doi.org/10.1007/bf02900865

1996-01-01

Wuhan University Journal of Natural Sciences

Abstract:An object-oriented C++ parallel compiler system, called OOCPCS, is developed to facilitate programmers to write sequential programs using C++ or Annotated C++ language for parallel computation. OOCPCS bases on an integrated object-oriented paradigm and large-grain data flow model, called OOLGDFM, and recognizes automatically parallel objects using parallel compiling techniques. The paper describes the object-oriented parallel model and realization of the system on networks.
An Automatic Parallelizer For Coarse-Grained Reconfigurable Processor

Ping Mi,Zhongyuan Zhao,Weiguang Sheng,Weifeng He

DOI: https://doi.org/10.1109/ICSICT.2016.7998880

2016-01-01

Abstract:Coarse-Grained Reconfigurable Architectures (CGRA) can accelerate computing speed with high power efficiency. Based on its special architecture, a corresponding compiler is designed to map the applications onto CGRAs. In order to exploit its parallelism, we design an automatic parallelizer for the compiler. This tool is aimed to transform source code to target code with multiple sub-functions, which has significant impact on the computing performance. In this paper, an algorithm is developed for code transformation and a parallel scheduling policy is applied. Our experiments show that this module can improve the computing performance.
Exploring Object-Level Parallelism on Chip Multi-Processors

Weixing Ji,Yizhuo Wang,Zhi Huang,Junqing Zhao,Xi Li

DOI: https://doi.org/10.1007/978-3-642-33065-0_9

2012-01-01

Abstract:Object-oriented programming languages are prevalent at present and the inherent parallelism of a wide range of applications can be expressed as groups of concurrent objects. This paper proposes a new object-level parallel programming model to meet the challenge of parallel programming on chip multi-processors. This object-based computation model maps multiple software objects onto different physical processor cores at runtime. Independent objects can run concurrently and communicate with each other by synchronous, asynchronous, future and broadcast messages. In order to verify the efficiency of this model, a library implementation CPPLib of this model based on C++ is presented. The library implementation of this model requires no extension to existing object-oriented programming languages and provides good salability and portability. The performance of CPPLib is evaluated on both a dual-core desktop computer and an 8-core server using standard benchmarks. Experimental results show that CPPLib achieves average speedup levels of 1.5 and 3.6 on the dual-core desktop computer and the 8-core server, respectively, compared with their serial version.
AUTOPARLLM: GNN-Guided Automatic Code Parallelization using Large Language Models

Quazi Ishtiaque Mahmud,Ali TehraniJamsaz,Hung D Phan,Nesreen K. Ahmed,Ali Jannesari

2023-10-09

Abstract:Parallelizing sequentially written programs is a challenging task. Even experienced developers need to spend considerable time finding parallelism opportunities and then actually writing parallel versions of sequentially written programs. To address this issue, we present AUTOPARLLM, a framework for automatically discovering parallelism and generating the parallel version of the sequentially written program. Our framework consists of two major components: i) a heterogeneous Graph Neural Network (GNN) based parallelism discovery and parallel pattern detection module, and ii) an LLM-based code generator to generate the parallel counterpart of the sequential programs. We use the GNN to learn the flow-aware characteristics of the programs to identify parallel regions in sequential programs and then construct an enhanced prompt using the GNN's results for the LLM-based generator to finally produce the parallel counterparts of the sequential programs. We evaluate AUTOPARLLM on 11 applications of 2 well-known benchmark suites: NAS Parallel Benchmark and Rodinia Benchmark. Our results show that AUTOPARLLM is indeed effective in improving the state-of-the-art LLM-based models for the task of parallel code generation in terms of multiple code generation metrics. AUTOPARLLM also improves the average runtime of the parallel code generated by the state-of-the-art LLMs by as high as 3.4% and 2.9% for the NAS Parallel Benchmark and Rodinia Benchmark respectively. Additionally, to overcome the issue that well-known metrics for translation evaluation have not been optimized to evaluate the quality of the generated parallel code, we propose OMPScore for evaluating the quality of the generated code. We show that OMPScore exhibits a better correlation with human judgment than existing metrics, measured by up to 75% improvement of Spearman correlation.

Machine Learning
Automatic code generation of subword parallelism based on loop unrolling

Wang Miao,Wang Zhiying,Shen Li,Dai Kui

DOI: https://doi.org/10.3321/j.issn:1671-4512.2008.02.012

2008-01-01

Abstract:Well-known parallelization techniques can be used to exploit subword parallelism. Loop unrolling, register renaming and induction variable expansion prove to be valuable to achieve this goal. We evaluated the performance of the code generated by our method for a number of benchmarks. The results reveal that our compiler produces a performance improvement over the code generated without the subword parallelism.
A Language of Suggestions for Program Parallelization

Chao Zhang,Chen Ding,Kirk Kelsey,Tongxin Bai,Xiaoming Gu,Xiaobing Feng

2009-01-01

Abstract:Coarse-grained task parallelism exists in sequential code and can be leveraged to boost the use of chip multi-processors. However, large tasks may execute thousands of lines of code and are often too complex to analyze and manage statically. This report describes a programming system called suggestible parallelization. It consists of a programming in- terface and a support system. The interface is a small language with three primitives for marking possibly parallel tasks and their possible dependences. The support system is implemented in software and ensures correct parallel execution through speculative par- allelization, speculative communication and speculative memory allocation. It manages parallelism dynamically to tolerate unevenness in task size, inter-task delay and hardware speed. When evaluated using four full-size benchmark applications, suggestible parallelization obtains up to a 6 times speedup over 10 processors for sequential legacy applications up to 35 thousand lines in size. The overhead of software speculation is not excessively high compared to unprotected parallel execution.
Parallel Programming Models and Languages

安虹,陈国良

2002-01-01

Journal of Software

Abstract:Parallel computing is about 20 years old. Till now, there is a lack of effective parallel programming methods and tools in high performance computing as a result that it is very difficulty to parallel programming, understanding behaviors of parallel program, debugging parallel codes and optimizing performance. In this paper, the reasons why so difficulty parallel programming is are analyzed, while the issues about parallel programming methods existing in recent high performance parallel machines are addressed, the current status of parallel programming models and languages are surveyed, the view of the criteria is offered to evaluate parallel programming models, the challenge problems in this area are brought forward, and some future research directions are pointed out.
GPT-Driven Source-to-Source Transformation for Generating Compilable Parallel CUDA Code for Nussinov's Algorithm

Marek Palkowski,Mateusz Gruzewski

DOI: https://doi.org/10.3390/electronics13030488

IF: 2.9

2024-01-25

Electronics

Abstract:Designing automatic optimizing compilers is an advanced engineering process requiring a great deal of expertise, programming, testing, and experimentation. Maintaining the approach and adapting it to evolving libraries and environments is a time-consuming effort. In recent years, OpenAI has presented the GPT model, which is designed for many fields like computer science, image processing, linguistics, and medicine. It also supports automatic programming and translation between programming languages, as well as human languages. This article will verify the usability of the commonly known LLM model, GPT, for the non-trivial NPDP Nussinov's parallel algorithm code within the OpenMP standard to create a parallel equivalent of CUDA for NVIDIA graphics cards. The goal of this approach is to avoid creating any post-processing scripts and writing any lines of target code. To validate the output code, we compare the resulting arrays with the ones calculated by the optimized code for the CPU generated employing the polyhedral compilers. Finally, the code will be checked for scalability and performance. We will concentrate on assessing the capabilities of GPT, highlighting common challenges that can be refined during future learning processes. This will enhance code generation for various platforms by leveraging the outcomes from polyhedral optimizers.

engineering, electrical & electronic,computer science, information systems,physics, applied
APR: A Novel Parallel Repacking Algorithm for Efficient GPGPU Parallel Code Transformation.

Yulong Yu,Xubin He,He Guo,Sihui Zhong,Yuxin Wang,Xin Chen,Weijun Xiao

DOI: https://doi.org/10.1145/2576779.2576789

2014-01-01

Abstract:General-purpose graphics processing units (GPGPU) brings an opportunity to improve the performance for many applications. However, exploiting parallelism is low productive in current programming frameworks such as CUDA and OpenCL. Programmers have to consider and deal with many GPGPU architecture details; therefore it is a challenge to trade off the programmability and the efficiency of performance tuning. Parallel Repacking (PR) is a popular performance tuning approach for GPGPU applications, which improves the performance by changing the parallel granularity. Existing code transformation algorithms using PR increase the productivity, but they do not cover adequate code patterns and do not give an effective code error detection. In this paper, we propose a novel parallel repacking algorithm (APR) to cover a wide range of code patterns and improve efficiency. We develop an efficient code model that expresses a GPGPU program as a recursive statement sequence, and introduces a concept of singular statement. APR building upon this model uses appropriate transformation rules for singular and non-singular statements to generate the repacked codes. A recursive transformation is performed when it encounters a branching/loop singular statement. Additionally, singular statements unify the transformation for barriers and data sharing, and enable APR to detect the barrier errors. The experiment results based on a prototype show that out proposed APR covers more code patterns than existing solutions such as the automatic thread coarsening in Crest, and the repacked codes using the APR achieve effective performance gain up to 3.28X speedup, in some cases even higher than manually tuned repacked codes.
Dependence-Based Code Transformation for Coarse-Grained Parallelism

Bo Zhao,Zhen Li,Ali Jannesari,Felix Wolf,Weiguo Wu

DOI: https://doi.org/10.1145/2723772.2723777

2015-01-01

Abstract:Multicore architectures are becoming more common today. Many software products implemented sequentially have failed to exploit the potential parallelism of multicore architectures. Significant re-engineering and refactoring of existing software is needed to support the use of new hardware features. Due to the high cost of manual transformation, an automated approach to transforming existing software and taking advantage of multicore architectures would be highly beneficial. We propose a novel auto-parallelization approach, which integrates data-dependence profiling, task parallelism extraction and source-to-source transformation. Coarse-grained task parallelism is detected based on a concept called Computational Unit(CU). We use dynamic profiling information to gather control- and data-dependences among tasks and generate a task graph. In addition, we develop a source-to-source transformation tool based on LLVM, which can perform high-level code restructuring. It transforms the generated task graph with loop parallelism and task parallelism of sequential code into parallel code using Intel Threading Building Blocks (TBB). We have evaluated NAS Parallel Benchmark applications, three applications from PARSEC benchmark suite, and real world applications. The obtained results confirm that our approach is able to achieve promising performance with minor user interference. The average speedups of loop parallelization and task parallelization are 3.12x and 9.92x respectively.
Parallel pattern compiler for automatic global optimizations

Adrian Schmitz,Semih Burak,Julian Miller,Matthias S. Müller

DOI: https://doi.org/10.1016/j.parco.2024.103112

IF: 0.983

2024-09-22

Parallel Computing

Abstract:High-performance computing (HPC) systems enable scientific advances through simulation and data processing. The heterogeneity in HPC hardware and software increases the application complexity and reduces its maintainability and productivity. This work proposes a prototype implementation for a parallel pattern-based source-to-source compiler to address these challenges. The prototype limits the complexity of parallelism and heterogeneous architectures to parallel patterns that are optimized toward a given target architecture. By applying high-level optimizations and a mapping between parallel patterns and execution units during compile time, portability between systems is achieved. The compiler can address architectures with shared memory, distributed memory, and accelerator offloading. The approach shows speedups for seven of the nine supported Rodinia benchmarks, reaching speedups of up to twelve times. Porting LULESH to the Parallel Pattern Language (PPL) shows a compression of code size by 65% (3.4 thousand lines of code) through a more concise expression and a higher level of abstraction. The tool's limitations include dynamic algorithms that are challenging to analyze statically and overheads during the compile time optimization. This paper is an extended version of a previous PMAM publication (Schmitz et al., 2024).

computer science, theory & methods
Design considerations and test results of AFT - a new generation parallelizing compiler

ChuanQi Zhu,Binyu Zang,Tong Chen

DOI: https://doi.org/10.1109/APDC.1997.574064

1997-01-01

Abstract:An effective automatic parallelizer is critical for users to exploit the resources of parallel computers. Research has gained much progress in recent years. This paper introduces AFT, a new generation of parallelizing compiler that we have developed. It integrates many advanced techniques in an effective and efficient system. The experimental results show that AFT is able to achieve notable parallelization on many programs
Integrated optimization scheme in parallelizing compilers

Sun Tong,Li Sanli,Li Xiaoming

1996-01-01

Ruan Jian Xue Bao/Journal of Software

Abstract:This paper presents a complete suit of systematic optimizing methods which may be used in parallelizing compilers for multicomputers or computer clusters. In the compilation scheme, two strategy are adopted. One is trading off parallelism and communication cost and the other is reducing and hiding communication overhead. Through analyzing the properties of data communication required for the program partition approach based on affine functions, the authors find a method to exploit parallelism in aerial programs satisfying the special requirements of distributed memory machines. In order to minimize the total of data needing to be communicated, they invent a global optimization program partition method based on solving linear equations. In order to optimize the organization of communication codes and generate more efficient node programs, they invent a more practical method based on linear inequalities to perform communication optimization and node programs generation.
A user-guided semi-automatic parallelization method and its implementation

Chuliang Weng,Zhongguo Chen,Xinda Lu,Minglu Li,Yong Yin

DOI: https://doi.org/10.1007/11573937_23

2005-01-01

Abstract:In this paper, we propose a user-guided semi-automatic parallelization method, which is based on code templates corresponding to parallel programming paradigms and the concept of meta-task independent with each other. As an implementation of this method, we develop the system Metaparallel, which is based on Java language and MPICH, and the framework of Metaparallel is discussed. At last, the parallelization flow is studied with a case. In addition, we test the usability of Metaparallel by the practical engineering problem.
On the Parallelization Optimization Strategy for High Performance Computing Software

Weile Jia,Xiaodong Shi,Haifeng Lyu

2013-01-01

Abstract:With the arrival of the multi-core era, traditional software can barely utilize the peak performance of the hardware. Parallelization and optimization of the industrial code is a tough problem the HPC community is facing. In this paper, we present our parallel im-plementation using MPI, OpenMP and CUDA programming models. Different methods, implementation skills and optimization strategies are introduced. At the end, the challenge and the vision of the future work of our optimization strategy are discussed.

NUAPC: A Parallelizing Compiler for C++

Design, Implementation of the Parallel C Language Based on C/S Mode in Distributed Systems

A New Methodology of Data Dependence Analysis for Parallelizing C++

A Path-Based Method of Parallelizing C++ Programs

Hybrid Performance Modeling And Analyzing Of Parallel Systems

Shared Variable Oriented Parallel Precompiler for SPMD Model.

An Object Oriented C++ Parallel Compiler System

An Automatic Parallelizer For Coarse-Grained Reconfigurable Processor

Exploring Object-Level Parallelism on Chip Multi-Processors

AUTOPARLLM: GNN-Guided Automatic Code Parallelization using Large Language Models

Automatic code generation of subword parallelism based on loop unrolling

A Language of Suggestions for Program Parallelization

Parallel Programming Models and Languages

GPT-Driven Source-to-Source Transformation for Generating Compilable Parallel CUDA Code for Nussinov's Algorithm

APR: A Novel Parallel Repacking Algorithm for Efficient GPGPU Parallel Code Transformation.

Dependence-Based Code Transformation for Coarse-Grained Parallelism

Parallel pattern compiler for automatic global optimizations

Design considerations and test results of AFT - a new generation parallelizing compiler

Integrated optimization scheme in parallelizing compilers

A user-guided semi-automatic parallelization method and its implementation

On the Parallelization Optimization Strategy for High Performance Computing Software