Abstract:Throughout many fields, practitioners often rely on differential equations to model systems. Yet, for many applications, the theoretical derivation of such equations and/or accurate resolution of their solutions may be intractable. Instead, recently developed methods, including those based on parameter estimation, operator subset selection, and neural networks, allow for the data-driven discovery of both ordinary and partial differential equations (PDEs), on a spectrum of interpretability. The success of these strategies is often contingent upon the correct identification of representative equations from noisy observations of state variables and, as importantly and intertwined with that, the mathematical strategies utilized to enforce those equations. Specifically, the latter has been commonly addressed via unconstrained optimization strategies. Representing the PDE as a neural network, we propose to discover the PDE by solving a constrained optimization problem and using an intermediate state representation similar to a Physics-Informed Neural Network (PINN). The objective function of this constrained optimization problem promotes matching the data, while the constraints require that the PDE is satisfied at several spatial collocation points. We present a penalty method and a widely used trust-region barrier method to solve this constrained optimization problem, and we compare these methods on numerical examples. Our results on the Burgers' and the Korteweg-De Vreis equations demonstrate that the latter constrained method outperforms the penalty method, particularly for higher noise levels or fewer collocation points. For both methods, we solve these discovered neural network PDEs with classical methods, such as finite difference methods, as opposed to PINNs-type methods relying on automatic differentiation. We briefly highlight other small, yet crucial, implementation details.

What problem does this paper attempt to address?

The paper attempts to address the problem of discovering partial differential equations (PDE) from noisy data. Specifically, the authors propose a new strategy based on constrained optimization methods to discover PDEs, which differs from traditional unconstrained optimization or penalty methods. The core objective of the paper is to more accurately identify representative equations when dealing with noisy observational data and to ensure that these equations are satisfied at multiple spatial discretization points through constrained optimization. The main contributions of the paper are: 1. **Improved Noise Robustness**: By using constrained optimization methods (such as the trust-region interior point method), the accuracy of discovering PDEs under high noise levels is improved. 2. **Avoiding Overfitting**: By simultaneously fitting the data and discovering the PDE, robustness to noise and other implementation details is increased. 3. **Comparison of Different Methods**: The paper compares penalty methods with the trust-region interior point method and demonstrates that the latter performs better under high noise levels. Overall, this paper aims to explore the use of more sophisticated constrained optimization methods in scientific machine learning to improve the accuracy and robustness of discovering PDEs from data.

Constrained or Unconstrained? Neural-Network-Based Equation Discovery from Data

Discovering Physics-Informed Neural Networks Model for Solving Partial Differential Equations through Evolutionary Computation

An invariance constrained deep learning network for PDE discovery

PDE-constrained Models with Neural Network Terms: Optimization and Global Convergence

An invariance constrained deep learning network for partial differential equation discovery

Characterizing and Mitigating the Difficulty in Training Physics-informed Artificial Neural Networks under Pointwise Constraints

A Combination of Deep Neural Networks and Physics to Solve the Inverse Problem of Burger's Equation

Deep-learning of parametric partial differential equations from sparse and noisy data

Discovering Governing Equations by Machine Learning implemented with Invariance

A hybrid physics-informed neural network based multiscale solver as a partial differential equation constrained optimization problem

Constrained Neural Ordinary Differential Equations with Stability Guarantees

Solving Partial Differential Equations Using Deep Learning and Physical Constraints

Learning To Solve Differential Equation Constrained Optimization Problems

Physics-constrained robust learning of open-form partial differential equations from limited and noisy data

A PINN Approach to Symbolic Differential Operator Discovery with Sparse Data

Partial-differential-algebraic equations of nonlinear dynamics by Physics-Informed Neural-Network: (I) Operator splitting and framework assessment

Physics Constrained Learning for Data-driven Inverse Modeling from Sparse Observations

Learning Partial Differential Equations from Data Using Neural Networks

A Stable and Scalable Method for Solving Initial Value PDEs with Neural Networks

dNNsolve: an efficient NN-based PDE solver

Grad-Shafranov equilibria via data-free physics informed neural networks