Abstract:AI integration is revolutionizing the landscape of HPC simulations, enhancing the importance, use, and performance of AI-driven HPC workflows. This paper surveys the diverse and rapidly evolving field of AI-driven HPC and provides a common conceptual basis for understanding AI-driven HPC workflows. Specifically, we use insights from different modes of coupling AI into HPC workflows to propose six execution motifs most commonly found in scientific applications. The proposed set of execution motifs is by definition incomplete and evolving. However, they allow us to analyze the primary performance challenges underpinning AI-driven HPC workflows. We close with a listing of open challenges, research issues, and suggested areas of investigation including the the need for specific benchmarks that will help evaluate and improve the execution of AI-driven HPC workflows.

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is to couple artificial intelligence (AI) with high - performance computing (HPC) workflows to significantly improve the efficiency and performance of scientific computing. Specifically, the paper aims to: 1. **Improve the effective performance of HPC workflows**: By coupling the AI system to HPC simulations in real - time, enabling AI to guide or influence HPC tasks and vice versa, thereby enhancing the efficiency of "scientific research with given computing resources". 2. **Overcome the limitations of traditional forward simulations**: AI - coupled HPC workflows can overcome the bottlenecks of traditional simulations in physical and time scales or at higher resolutions in a more complex and extensive way. 3. **Provide sustainable and scalable performance gains**: By integrating AI into the computing workflow, significant performance improvements can be obtained without relying on specific processor architectures. 4. **Propose execution motifs**: By analyzing the applications of AI - HPC coupling in different modes, six of the most common execution motifs are identified, which help to understand the main performance challenges of AI - driven HPC workflows. 5. **Discuss open research questions and challenges**: This includes the need for specific benchmark tests to evaluate and improve the execution of AI - driven HPC workflows, and how to deal with the coupling problem between large - scale AI model training and HPC simulations. ### Six Execution Motifs 1. **AI - based Steering Ensembles of Simulations** - **Interaction mode**: Data flows from HPC to AI, and control flows from AI to HPC. - **Coupling mode**: Near - real - time analysis, dynamically generate or terminate simulations. - **Scope**: Use AI inference to improve HPC components online. 2. **Multistage Pipeline** - **Interaction mode**: Data flows from one HPC stage to multiple AI analysis components, and AI controls the execution of the next HPC stage. - **Coupling mode**: Near - real - time interaction, but the process usually does not dynamically generate or terminate. - **Scope**: Mainly used for branching the execution of HPC to optimize results. 3. **Inverse Design** - **Interaction mode**: Multiple HPC simulations send data to AI, and AI controls by iteratively optimizing design parameters. - **Coupling mode**: Static, concurrent or asynchronous execution, usually within the same system. - **Scope**: Improve both AI and HPC components simultaneously. 4. **Digital Replica** - **Interaction mode**: Data and control flow bidirectionally, combining experiments and HPC with AI. - **Coupling mode**: Real - time requirements, monitoring and visualization, and there may be scientists involved in adjusting future runs. - **Scope**: AI and HPC improve each other. 5. **Distributed Models and Dynamic Data** - **Interaction mode**: Data and control flow bidirectionally, involving multiple HPC and AI components. - **Coupling mode**: Geographically distributed, adapting to wide - area network performance. - **Scope**: AI is used to compress, reconstruct or remotely visualize dynamic data. 6. **Adaptive Training** - **Interaction mode**: Data flows from HPC to multiple AIs, and control flows from analysis to HPC and AI training. - **Coupling mode**: Near - real - time requirements, dynamic combination. - **Scope**: HPC is used to improve AI models. ### Summary By identifying and describing these execution motifs, the paper aims to provide a common conceptual basis for AI - HPC - coupled workflows, help understand their main performance challenges, and point the way for future research and development.

AI-coupled HPC Workflow Applications, Middleware and Performance

P F ] 1 3 A ug 2 01 9 HPC AI 500 : A Benchmark Suite for HPC AI Systems

HPC AI500: A Benchmark Suite for HPC AI Systems

AI-oriented Workload Allocation for Cloud-Edge Computing.

Data Motifs

Towards Seamless Management of AI Models in High-Performance Computing

Convergence of Artificial Intelligence and High Performance Computing on NSF-supported Cyberinfrastructure

A Comprehensive Evaluation of Novel AI Accelerators for Deep Learning Workloads

Transforming the Hybrid Cloud for Emerging AI Workloads

Infrastructure for Artificial Intelligence, Quantum and High Performance Computing

SAIH: A Scalable Evaluation Methodology for Understanding AI Performance Trend on HPC Systems

Employing artificial intelligence to steer exascale workflows with colmena

AIPerf: Automated machine learning as an AI-HPC benchmark

Cloud Services Enable Efficient AI-Guided Simulation Workflows across Heterogeneous Resources

Integrating Deep Learning in Domain Sciences at Exascale

Advancing parallel programming integrating artificial intelligence for enhanced efficiency and automation

Hyperparameter optimization, quantum-assisted model performance prediction, and benchmarking of AI-based High Energy Physics workloads using HPC

Evaluating Emerging AI/ML Accelerators: IPU, RDU, and NVIDIA/AMD GPUs

Performance on HPC Platforms Is Possible Without C++

Workflows for Artificial Intelligence