Abstract:The International Journal of High Performance Computing Applications, Ahead of Print. Ytopt is a Python machine-learning-based autotuning software package developed within the ECP PROTEAS-TUNE project. The ytopt software adopts an asynchronous search framework that consists of sampling a small number of input parameter configurations and progressively fitting a surrogate model over the input-output space until exhausting the user-defined maximum number of evaluations or the wall-clock time. libEnsemble is a Python toolkit for coordinating workflows of asynchronous and dynamic ensembles of calculations across massively parallel resources developed within the ECP PETSc/TAO project. libEnsemble helps users take advantage of massively parallel resources to solve design, decision, and inference problems and expands the class of problems that can benefit from increased parallelism. In this paper we present our methodology and framework to integrate ytopt and libEnsemble to take advantage of massively parallel resources to accelerate the autotuning process. Specifically, we focus on using the proposed framework to autotune the ECP ExaSMR application OpenMC, an open source Monte Carlo particle transport code. OpenMC has seven tunable parameters some of which have large ranges such as the number of particles in-flight, which is in the range of 100,000 to 8 million, with its default setting of 1 million. Setting the proper combination of these parameter values to achieve the best performance is extremely time-consuming. Therefore, we apply the proposed framework to autotune the MPI/OpenMP offload version of OpenMC based on a user-defined metric such as the figure of merit (FoM) (particles/s) or energy efficiency energy-delay product (EDP) on Crusher at Oak Ridge Leadership Computing Facility. The experimental results show that we achieve the improvement up to 29.49% in FoM and up to 30.44% in EDP.

IOPathTune: Adaptive Online Parameter Tuning for Parallel File System I/O Path

An End-to-end and Adaptive I/O Optimization Tool for Modern HPC Storage Systems

Time-sharing Parallel Applications Through Performance-Targeted Feedback-Controlled Real-Time Scheduling.

I/O Bottleneck Detection and Tuning: Connecting the Dots using Interactive Log Analysis

Performance-Aware Data Placement in Hybrid Parallel File Systems.

Output Performance Study on a Production Petascale Filesystem.

Boosting Parallel File System Performance Via Heterogeneity-Aware Selective Data Layout.

Tarazu: An Adaptive End-to-End I/O Load Balancing Framework for Large-Scale Parallel File Systems

ytopt: Autotuning Scientific Applications for Energy Efficiency at Large Scales

vPFS+: Managing I/O Performance for Diverse HPC Applications

Enhancing Online Index Tuning with a Learned Tuning Diagnostic.

Parallel computing based parameter auto-tuning algorithm for optimization solvers

A Two-Level Parallel Decomposition Approach for Transient Stability Constrained Optimal Power Flow

A performance optimization framework for hardware/software co design

Integrating ytopt and libEnsemble to autotune OpenMC

Accelerating Filesystem Checking and Repair with pFSCK

HAS: Heterogeneity-Aware Selective Data Layout Scheme for Parallel File Systems on Hybrid Servers

Towards Enabling I/O Awareness in Task-based Programming Models

HARL: Optimizing Parallel File Systems with Heterogeneity-Aware Region-Level Data Layout

Cost-Effective Methodology for Complex Tuning Searches in HPC: Navigating Interdependencies and Dimensionality

A Case Study on the Efficiency of User-Level Parallel File Systems