François PacaudMichel SchanenSungho ShinDaniel Adrian MaldonadoMihai Anitescua Centre Automatique et Systèmes,Mines Paris–PSL,Paris,Franceb Mathematics and Computer Science Department,Argonne National Laboratory,Lemont,IL,USAFrançois Pacaud is an assistant professor at Mines Paris-PSL,in the Centre Automatique et Systèmes (CAS). He obtained his M.Sc. in 2015 from Mines Paris-PSL,and in 2018 his Ph.D. in applied mathematics from the École des Ponts ParisTech,Paris,France. He was a postdoctoral fellow in the Compute Science Division at Argonne National Laboratory between 2020 and 2022. He his an expert in nonlinear optimization and numerical analysis for energy systems.Michel Schanen obtained is Ph.D. in automatic differentiation of MPI simulations from RWTH Aachen University,Germany. He joined Argonne as a Postdoctoral Appointee investigating large-scale adjoints in the fluid dynamics code Nek5000. In 2017,his research changed to optimization for power grid simulations as a computational engineer. His interests revolve around using the programming language Julia for modeling and running simulations efficiently on leadership computing systems.Sungho Shin received a B.S. degree in chemical engineering and mathematics from Seoul National University,Seoul,South Korea,in 2016. He received a Ph.D. degree in chemical engineering from the University of Wisconsin-Madison,Madison,WI,USA,in 2021. He is currently a postdoctoral researcher at the Mathematics and Computer Science Division of Argonne National Laboratory. His research interests include control theory and optimization algorithms for energy systems.Daniel Adrian Maldonado received the B.S. degree from the Universitat Politecnica de Valencia,Valencia,Spain,and the M.Eng. and Ph.D. degree from the Illinois Institute of Technology,Chicago. His research interests include transient stability simulation,state estimation,and optimal control.Mihai Anitescu is a senior computational mathematician in the Mathematics and Computer Science Division at Argonne National Laboratory and a professor in the Department of Statistics at the University of Chicago. He obtained his engineer diploma (electrical engineering) from the Polytechnic University of Bucharest in 1992 and his Ph.D. in applied mathematical and computational sciences from the University of Iowa in 1997. He specializes in the areas of numerical optimization,computational science,numerical analysis,and uncertainty quantification. He is on the editorial board of the SIAM Journal on Optimization,and he is a senior editor for Optimization Methods and Software. He is a past member of the editorial boards of Mathematical Programming A and B,SIAM Journal on Scientific Computing,and SIAM/ASA Journal in Uncertainty Quantification.

Abstract:We investigate how to port the standard interior-point method to new exascale architectures for block-structured nonlinear programs with state equations. Computationally, we decompose the interior-point algorithm into two successive operations: the evaluation of the derivatives and the solution of the associated Karush-Kuhn-Tucker (KKT) linear system. Our method accelerates both operations using two levels of parallelism. First, we distribute the computations on multiple processes using coarse parallelism. Second, each process uses SIMD/GPU accelerators locally to accelerate the operations using fine-grained parallelism. The KKT system is reduced by eliminating the inequalities and the state variables from the corresponding equations. We demonstrate our method's capability on the supercomputer Polaris, a testbed for the future exascale Aurora system. Each node is equipped with four GPUs, a setup amenable to our two-level approach. Our experiments on the stochastic optimal power flow problem show that the reduction method is 50x faster than the sparse linear solver HSL MA57 running in serial on the CPU, and 6x faster than Pardiso running in parallel on CPU on the same number of processes.

Batched Second-Order Adjoint Sensitivity for Reduced Space Methods

Condensed interior-point methods: porting reduced-space approaches on GPU hardware

A scalable multi-GPU method for semi-implicit fractional-step integration of incompressible Navier-Stokes equations

Accelerating Condensed Interior-Point Methods on SIMD/GPU Architectures

Condensed-space methods for nonlinear programming on GPUs

SGN: Sparse Gauss-Newton for Accelerated Sensitivity Analysis

Scalable Gradients for Stochastic Differential Equations

Fast and Scalable FFT-Based GPU-Accelerated Algorithms for Hessian Actions Arising in Linear Inverse Problems Governed by Autonomous Dynamical Systems

Parallel interior-point solver for block-structured nonlinear programs on SIMD/GPU architectures

An AD based library for Efficient Hessian and Hessian-Vector Product Computation on GPU

Implicit method for the solution of supersonic and hypersonic 3D flow problems with Lower-Upper Symmetric-Gauss-Seidel preconditioner on multiple graphics processing units

A C++ implementation of the discrete adjoint sensitivity analysis method for explicit adaptive Runge-Kutta methods enabled by automatic adjoint differentiation and SIMD vectorization

AdjointDEIS: Efficient Gradients for Diffusion Models

Level set topology optimization with sparse automatic differentiation

Improving the Efficiency of Gradient Descent Algorithms Applied to Optimization Problems with Dynamical Constraints

A discrete adjoint gradient approach for equality and inequality constraints in dynamics

A Duality-Preserving Adjoint Method for Segregated Navier–Stokes Solvers

A Comparison of Automatic Differentiation and Continuous Sensitivity Analysis for Derivatives of Differential Equation Solutions

An adaptive finite element multigrid solver using GPU acceleration

Adjoint sensitivity in PDE constrained least squares problems as a multiphysics problem

Efficient parallelization strategy for real-time FE simulations