Abstract:The recent dramatic progress in machine learning is partially attributed to the availability of high-performant computers and development tools. The accelerated linear algebra (XLA) compiler is one such tool that automatically optimises array operations (mostly fusion to reduce memory operations) and compiles the optimised operations into high-performant programs specific to target computing platforms. Like machine-learning models, numerical models are often expressed in array operations, and thus their performance can be boosted by XLA. This study is the first of its kind to examine the efficiency of XLA for numerical models, and the efficiency is examined stringently by comparing its performance with that of optimal implementations. Two shared-memory computing platforms are examined-the CPU platform and the GPU platform. To obtain optimal implementations, the computing speed and its optimisation are rigorously studied by considering different workloads and the corresponding computer performance. Two simple equations are found to faithfully modell the computing speed of numerical models with very few easily-measureable parameters. Regarding operation optimisation within XLA, results show that models expressed in low-level operations (e.g., slice, concatenation, and arithmetic operations) are successfully fused while high-level operations (e.g., convolution and roll) are not. Regarding compilation within XLA, results show that for the CPU platform of certain computers and certain simple numerical models on the GPU platform, XLA achieves high efficiency (> 80%) for large problems and acceptable efficiency (10%~80%) for medium-size problems-the gap is from the overhead cost of Python. Unsatisfactory performance is found for the CPU platform of other computers (operations are compiled in a non-optimal way) and for high-dimensional complex models for the GPU platform, where each GPU thread in XLA handles 4 (single precision) or 2 (double precision) output elements-hoping to exploit the high-performant instructions that can read/write 4 or 2 floating-point numbers with one instruction. However, these instructions are rarely used in the generated code for complex models and performance is negatively affected. Therefore, flags should be added to control the compilation for these non-optimal scenarios.

GreenLA: green linear algebra software for GPU-accelerated heterogeneous computing.

Lasa: Abstraction and Specialization for Productive and Performant Linear Algebra on FPGAs

HLanc: Heterogeneous Parallel Implementation of the Implicitly Restarted Lanczos Method

Lean Algebraic Multigrid (LAMG): Fast Graph Laplacian Linear Solver

Accelerated linear algebra compiler for computationally efficient numerical models: Success and potential area of improvement

Ginkgo: A Modern Linear Operator Algebra Framework for High Performance Computing

Analysis of Parallel Algorithms for Energy Conservation with GPU.

Towards a platform-portable linear algebra backend for OpenFOAM

Energy-Efficient Implementation of the Lattice Boltzmann Method

LAW: A Tool for Improved Productivity with High-Performance Linear Algebra Codes. Design and Applications

The ELAPS Framework: Experimental Linear Algebra Performance Studies

dMath: A Scalable Linear Algebra and Math Library for Heterogeneous GP-GPU Architectures

Energy efficiency: a Lattice Boltzmann study

Performance Portable Solid Mechanics via Matrix-Free $p$-Multigrid

Automatic Generation of Efficient Linear Algebra Programs

Porting a sparse linear algebra math library to Intel GPUs

Linnea: Automatic Generation of Efficient Linear Algebra Programs

Matrix-free approaches for GPU acceleration of a high-order finite element hydrodynamics application using MFEM, Umpire, and RAJA

MAGMA: Enabling exascale performance with accelerated BLAS and LAPACK for diverse GPU architectures

Generalized Gpu Acceleration For Applications Employing Finite-Volume Methods

AmgX: A Library for GPU Accelerated Algebraic Multigrid and Preconditioned Iterative Methods