Abstract:Multi-core shared memory architectures have become ubiquitous in computing hardware nowadays. As a result, there is a growing need to fully utilize these architectures by introducing appropriate parallelization schemes, such as OpenMP worksharing-loop constructs, to applications. However, most developers find introducing OpenMP directives to their code hard due to pervasive pitfalls in managing parallel shared memory. To assist developers in this process, many compilers, as well as source-to-source (S2S) translation tools, have been developed over the years, tasked with inserting OpenMP directives into code automatically. In addition to having limited robustness to their input format, these compilers still do not achieve satisfactory coverage and precision in locating parallelizable code and generating appropriate directives. Recently, many data-driven AI-based code completion (CC) tools, such as GitHub CoPilot, have been developed to ease and improve programming productivity. Leveraging the insights from existing AI-based programming-assistance tools, this work presents a novel AI model that can serve as a parallel-programming assistant. Specifically, our model, named PragFormer , is tasked with identifying for loops that can benefit from conversion to parallel worksharing-loop construct (OpenMP directive) and even predict the need for specific data-sharing attributes clauses on the fly. We created a unique database, named Open-OMP , specifically for this goal. Open-OMP contains over 32,000 unique code snippets from different domains, half of which contain OpenMP directives, while the other half do not. We experimented with different model design parameters for these tasks and showed that our best-performing model outperforms a statistically-trained baseline as well as a state-of-the-art S2S compiler. In fact, it even outperforms the popular generative AI model of ChatGPT. In the spirit of advancing research on this topic, we have already released source code for PragFormer as well as Open-OMP dataset to public. Moreover, an interactive demo of our tool, as well as a Hugging Face webpage to experiment with our tool, are already available.

PragFormer: Data-Driven Parallel Source Code Classification with Transformers

Advising OpenMP Parallelization via a Graph-Based Approach with Transformers

OMPar: Automatic Parallelization with AI-Driven Source-to-Source Compilation

OMP-Engineer: Bridging Syntax Analysis and In-Context Learning for Efficient Automated OpenMP Parallelization

GPT-Driven Source-to-Source Transformation for Generating Compilable Parallel CUDA Code for Nussinov's Algorithm

OMPGPT: A Generative Pre-trained Transformer Model for OpenMP

Learning to Parallelize with OpenMP by Augmented Heterogeneous AST Representation

MPIrigen: MPI Code Generation through Domain-Specific Language Models

CodeTrans: Towards Cracking the Language of Silicon's Code Through Self-Supervised Deep Learning and High Performance Computing

HPC-Coder: Modeling Parallel Programs using Large Language Models

AUTOPARLLM: GNN-Guided Automatic Code Parallelization using Large Language Models

Dependence-Based Code Transformation for Coarse-Grained Parallelism

Scope is all you need: Transforming LLMs for HPC Code

Evaluating AI-generated code for C++, Fortran, Go, Java, Julia, Matlab, Python, R, and Rust

Competition-level code generation with AlphaCode

OpenMP Advisor

ComPile: A Large IR Dataset from Production Sources

Compilable Neural Code Generation with Compiler Feedback

Should AI Optimize Your Code? A Comparative Study of Current Large Language Models Versus Classical Optimizing Compilers

ProGraML: Graph-based Deep Learning for Program Optimization and Analysis

Execution-based Code Generation using Deep Reinforcement Learning