Abstract:Most inverse problems from physical sciences are formulated as PDE-constrained optimization problems. This involves identifying unknown parameters in equations by optimizing the model to generate PDE solutions that closely match measured data. The formulation is powerful and widely used in many sciences and engineering fields. However, one crucial assumption is that the unknown parameter must be deterministic. In reality, however, many problems are stochastic in nature, and the unknown parameter is random. The challenge then becomes recovering the full distribution of this unknown random parameter. It is a much more complex task. In this paper, we examine this problem in a general setting. In particular, we conceptualize the PDE solver as a push-forward map that pushes the parameter distribution to the generated data distribution. This way, the SDE-constrained optimization translates to minimizing the distance between the generated distribution and the measurement distribution. We then formulate a gradient-flow equation to seek the ground-truth parameter probability distribution. This opens up a new paradigm for extending many techniques in PDE-constrained optimization to that for systems with stochasticity.

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve This paper aims to address the problem of recovering the distribution of random parameters from measurement data. Specifically, it focuses on how to infer the distribution of unknown random parameters in differential equations (DE) from the distribution of measurement data. ### Background and Motivation In physical sciences, many inverse problems can be formulated as optimization problems constrained by partial differential equations (PDE). These problems typically involve generating PDE solutions that closely match the measurement data by optimizing the model, thereby identifying the unknown parameters in the equations. However, traditional optimization methods assume these unknown parameters are deterministic. In reality, many problems exhibit randomness, and the unknown parameters are stochastic. Therefore, the challenge lies in recovering the full distribution of these unknown random parameters, which is a more complex problem. ### Main Contributions 1. **Gradient Flow Solver**: The paper proposes a first-order optimization solver based on gradient flow to solve DE-constrained optimization problems with randomness. By choosing different distance or divergence functions \( D \), corresponding gradient flows can be generated to move the parameter distribution \( \rho_u \). 2. **Particle Method**: When the distance function \( D \) is the Kullback-Leibler (KL) divergence and the metric of the probability space is the 2-Wasserstein distance, the paper provides a particle method to simulate the gradient flow equation. The update formula combines forward and adjoint solvers. 3. **Well-posedness Study in Linear Cases**: For linear problems, the paper studies the well-posedness in underdetermined and overdetermined cases and explores their relationship with the corresponding problems in deterministic cases. ### Numerical Experiments and Conclusions The paper validates the effectiveness of the proposed methods through numerical experiments and provides conclusions in the final section. ### Comparison with Related Research Areas 1. **Bayesian Inverse Problems**: Bayesian inverse problems use probabilistic models to infer unknown system parameters from observational data. The main difference from the framework in this paper is that the uncertainty in the Bayesian framework primarily arises from noise in the measurement process, whereas in this paper, the uncertainty is an inherent characteristic of the parameters themselves. 2. **Density Estimation**: Density estimation is a statistical technique used to estimate the probability density function of a random variable from observed samples. The framework in this paper degenerates to the classical density estimation problem when \( G = I \), but in general, it infers the distribution of parameters from the distribution of measurement data. ### Gradient Flow Formulas and Particle Methods The paper discusses gradient flow formulas under different metrics, including Wasserstein gradient flow, Hellinger gradient flow, etc., and proposes corresponding particle methods. These methods approximate the evolution of probability distributions through particle systems to achieve the optimization goal. ### Summary By introducing new gradient flow methods and particle methods, this paper provides an effective solution for DE-constrained optimization problems with randomness. These methods are not only theoretically innovative but also demonstrate good performance in practical applications.

Differential-Equation Constrained Optimization With Stochasticity

Differential Equation–Constrained Optimization with Stochasticity

Direct Approaches for PDE-constrained Dynamic Optimization Based on Space–time Orthogonal Collocation on Finite Elements

Parallel Stochastic Optimization Framework for Large-Scale Non-Convex Stochastic Problems

Stochastic Inverse Problem: stability, regularization and Wasserstein gradient flow

Global Optimization with Orthogonality Constraints Via Stochastic Diffusion on Manifold

Stochastic Successive Convex Approximation for Non-Convex Constrained Stochastic Optimization

Constrained or Unconstrained? Neural-Network-Based Equation Discovery from Data

A relaxation-based probabilistic approach for PDE-constrained optimization under uncertainty with pointwise state constraints

Learning To Solve Differential Equation Constrained Optimization Problems

The stochastic counterpart of conservation laws with heterogeneous conductivity fields: application to deterministic problems and uncertainty quantification

Physics Constrained Learning for Data-driven Inverse Modeling from Sparse Observations

Constrained Optimization with Decision-Dependent Distributions

Stochastic Optimization with Decision-Dependent Distributions

Potential in Stochastic Differential Equations: Novel Construction

Probability Contour Constrained Optimization and A Data-based Solution Paradigm

Convergence of the Euler scheme for stochastic functional partial differential equations

Stochastic Optimization for Non-convex Inf-Projection Problems

Parameter Estimation in Stochastic Differential Equations

Stochastic Differential Equations for Modeling First Order Optimization Methods

Control theory for stochastic distributed parameter systems, an engineering perspective