Abstract:PDE-constrained inverse problems are some of the most challenging and computationally demanding problems in computational science today. Fine meshes that are required to accurately compute the PDE solution introduce an enormous number of parameters and require large scale computing resources such as more processors and more memory to solve such systems in a reasonable time. For inverse problems constrained by time dependent PDEs, the adjoint method that is often employed to efficiently compute gradients and higher order derivatives requires solving a time-reversed, so-called adjoint PDE that depends on the forward PDE solution at each timestep. This necessitates the storage of a high dimensional forward solution vector at every timestep. Such a procedure quickly exhausts the available memory resources. Several approaches that trade additional computation for reduced memory footprint have been proposed to mitigate the memory bottleneck, including checkpointing and compression strategies. In this work, we propose a close-to-ideal scalable compression approach using autoencoders to eliminate the need for checkpointing and substantial memory storage, thereby reducing both the time-to-solution and memory requirements. We compare our approach with checkpointing and an off-the-shelf compression approach on an earth-scale ill-posed seismic inverse problem. The results verify the expected close-to-ideal speedup for both the gradient and Hessian-vector product using the proposed autoencoder compression approach. To highlight the usefulness of the proposed approach, we combine the autoencoder compression with the data-informed active subspace (DIAS) prior to show how the DIAS method can be affordably extended to large scale problems without the need of checkpointing and large memory.

Incremental data compression for PDE-constrained optimization with a data assimilation application

A Quasi-Sequential Algorithm for PDE-constrained Optimization Based on Space–time Orthogonal Collocation on Finite Elements

An Extended Iterative Format for the Progressive-Iteration Approximation.

IG-LSPIA: Least Squares Progressive Iterative Approximation for Isogeometric Collocation Method

An inexact variable metric variant of incremental aggregated Forward-Backward method for the large-scale composite optimization problem

A New Inexact Gradient Descent Method with Applications to Nonsmooth Convex Optimization

Online randomized interpolative decomposition with a posteriori error estimator for temporal PDE data reduction

Incremental Proper Orthogonal Decomposition Based Method for the Analyzation of Large Scale High Fidelity Simulation Results

An autoencoder compression approach for accelerating large-scale inverse problems

An Augmented Subspace Based Adaptive Proper Orthogonal Decomposition Method for Time Dependent Partial Differential Equations

Inexact Riemannian Gradient Descent Method for Nonconvex Optimization

On the Convergence of A Data-Driven Regularized Stochastic Gradient Descent for Nonlinear Ill-Posed Problems

Gradient Methods with Dynamic Inexact Oracles

Constrained Optimization with Compressed Gradients: A Dynamical Systems Perspective

MUSIC: Accelerated Convergence for Distributed Optimization With Inexact and Exact Methods

Accelerated Cyclic Coordinate Dual Averaging with Extrapolation for Composite Convex Optimization

Incremental Methods for Weakly Convex Optimization

Efficient mini-batch stochastic gradient descent with Centroidal Voronoi Tessellation for PDE-constrained optimization under uncertainty

An optimal randomized incremental gradient method