Abstract:(Partial) differential equations (PDEs) are fundamental tools for describing natural phenomena, making their solution crucial in science and engineering. While traditional methods, such as the finite element method, provide reliable solutions, their accuracy is often tied to the use of computationally intensive fine meshes. Moreover, they do not naturally account for measurements or prior solutions, and any change in the problem parameters requires results to be fully recomputed. Neural network-based approaches, such as physics-informed neural networks and neural operators, offer a mesh-free alternative by directly fitting those models to the PDE solution. They can also integrate prior knowledge and tackle entire families of PDEs by simply aggregating additional training losses. Nevertheless, they are highly sensitive to hyperparameters such as collocation points and the weights associated with each loss. This paper addresses these challenges by developing a science-constrained learning (SCL) framework. It demonstrates that finding a (weak) solution of a PDE is equivalent to solving a constrained learning problem with worst-case losses. This explains the limitations of previous methods that minimize the expected value of aggregated losses. SCL also organically integrates structural constraints (e.g., invariances) and (partial) measurements or known solutions. The resulting constrained learning problems can be tackled using a practical algorithm that yields accurate solutions across a variety of PDEs, neural network architectures, and prior knowledge levels without extensive hyperparameter tuning and sometimes even at a lower computational cost.

What problem does this paper attempt to address?

This paper attempts to solve the following problems: 1. **Limitations of traditional numerical methods**: Traditional methods for solving partial differential equations (PDEs), such as the finite element method (FEM), although providing reliable solutions, their accuracy usually depends on computationally intensive fine meshes. Moreover, these methods cannot naturally incorporate measurement data or previous solutions, and when the problem parameters change, the results need to be recalculated. 2. **Challenges of neural - network - based methods**: Neural - network - based methods (such as physics - informed neural networks (PINNs) and neural operators (NOs)) although providing mesh - free alternatives and being able to integrate prior knowledge, they are very sensitive to hyper - parameters (such as collocation points and loss weights), which limits their accuracy and applicability. To solve these problems, the paper proposes a Science - Constrained Learning (SCL) framework. Specifically, the main contributions of the paper are as follows: - **Proof of equivalence**: It is proved that finding the weak solution of a PDE is equivalent to solving a constrained learning problem with the worst - case loss. This finding explains the limitations of previous methods that minimize the aggregated value of the expected loss. - **Organic integration of structural and observational constraints**: The SCL framework can naturally combine structural constraints (such as invariance) and partial measurements or known solutions, thereby improving the generalization ability and accuracy of the model. - **Practical algorithm**: A hybrid sampling - optimization algorithm is proposed, which can effectively solve problems under various PDEs, neural network architectures, and prior knowledge levels without extensive hyper - parameter tuning, and sometimes even reduce the computational cost. Through these improvements, the SCL framework can provide more accurate PDE solutions under a wider range of conditions without relying on specific hyper - parameter selections or complex training techniques. ### Formula summary 1. **Integral equation in weak form**: \[ \int_{D} D_\pi[u](x, t) \varphi(x, t) \, dx \, dt = \int_{D} \tau(x, t) \varphi(x, t) \, dx \, dt, \quad \forall \varphi \in T \] where \(\varphi\) is the test function and \(T\) is usually the Sobolev space. 2. **Constrained learning problem**: \[ \min_{\theta \in \Theta} \mathbb{E}_{(x,t) \sim \psi_{BC}^\alpha} \left[ \left( u_\theta(x, t) - h(x, t) \right)^2 \right] \] subject to the following constraints: \[ \begin{aligned} &\mathbb{E}_{(x,t) \sim \psi_{PDE}^\alpha} \left[ \left( D[u_\theta](x, t) - \tau(x, t) \right)^2 \right] \leq \epsilon_{pde}, \\ &\mathbb{E}_{(x,t) \sim \psi_{ST}^\alpha} \left[ \left( u_\theta(x, t) - u_\theta[\gamma_i(x, t)] \right)^2 \right] \leq \epsilon_s, \quad \forall \gamma_i \in G, \\ &\mathbb{E}_{(x,t) \sim \psi_{OB}^\alpha} \left[ \left( u_\theta(x, t) - u_j^\dagger(x, t) \right)^2 \right] \leq \epsilon_o, \quad j = 1, \ldots, J. \end{aligned} \]

Solving Differential Equations with Constrained Learning

Learning To Solve Differential Equation Constrained Optimization Problems

Constrained or Unconstrained? Neural-Network-Based Equation Discovery from Data

Solving Differential Equation with Constrained Multilayer Feedforward Network.

Characterizing and Mitigating the Difficulty in Training Physics-informed Artificial Neural Networks under Pointwise Constraints

Spectral operator learning for parametric PDEs without data reliance

Physics Constrained Learning for Data-driven Inverse Modeling from Sparse Observations

An Unconstrained Formulation of Some Constrained Partial Differential Equations and its Application to Finite Neuron Methods

Probabilistic learning on manifolds constrained by nonlinear partial differential equations for small datasets

Learning a Neural Solver for Parametric PDE to Enhance Physics-Informed Methods

Solving Partial Differential Equations Using Deep Learning and Physical Constraints

Projected Neural Differential Equations for Learning Constrained Dynamics

Physics-constrained robust learning of open-form partial differential equations from limited and noisy data

Meta-PDE: Learning to Solve PDEs Quickly Without a Mesh

A Deep Learning Neural Network Framework for Solving Singular Nonlinear Ordinary Differential Equations

Solving Irregular and Data-enriched Differential Equations using Deep Neural Networks

PDE-constrained Models with Neural Network Terms: Optimization and Global Convergence

Solving partial differential equations with sampled neural networks

Hierarchical Learning to Solve Partial Differential Equations Using Physics-Informed Neural Networks

Solving PDEs on unknown manifolds with machine learning

Diffeomorphic Latent Neural Operators for Data-Efficient Learning of Solutions to Partial Differential Equations