Abstract:While physics-informed neural networks (PINNs) have been proven effective for low-dimensional partial differential equations (PDEs), the computational cost remains a hurdle in high-dimensional scenarios. This is particularly pronounced when computing high-order and high-dimensional derivatives in the physics-informed loss. Randomized Smoothing PINN (RS-PINN) introduces Gaussian noise for stochastic smoothing of the original neural net model, enabling Monte Carlo methods for derivative approximation, eliminating the need for costly auto-differentiation. Despite its computational efficiency in high dimensions, RS-PINN introduces biases in both loss and gradients, negatively impacting convergence, especially when coupled with stochastic gradient descent (SGD). We present a comprehensive analysis of biases in RS-PINN, attributing them to the nonlinearity of the Mean Squared Error (MSE) loss and the PDE nonlinearity. We propose tailored bias correction techniques based on the order of PDE nonlinearity. The unbiased RS-PINN allows for a detailed examination of its pros and cons compared to the biased version. Specifically, the biased version has a lower variance and runs faster than the unbiased version, but it is less accurate due to the bias. To optimize the bias-variance trade-off, we combine the two approaches in a hybrid method that balances the rapid convergence of the biased version with the high accuracy of the unbiased version. In addition, we present an enhanced implementation of RS-PINN. Extensive experiments on diverse high-dimensional PDEs, including Fokker-Planck, HJB, viscous Burgers', Allen-Cahn, and Sine-Gordon equations, illustrate the bias-variance trade-off and highlight the effectiveness of the hybrid RS-PINN. Empirical guidelines are provided for selecting biased, unbiased, or hybrid versions, depending on the dimensionality and nonlinearity of the specific PDE problem.

Robust Variational Physics-Informed Neural Networks

Collocation-based Robust Variational Physics-Informed Neural Networks (CRVPINN)

Variational Physics-Informed Neural Networks For Solving Partial Differential Equations

Effects of variational formulations on physics‐informed neural network performance in solid mechanics

Improving hp-Variational Physics-Informed Neural Networks for Steady-State Convection-Dominated Problems

Physics Informed Neural Networks with strong and weak residuals for advection-dominated diffusion problems

VW-PINNs: A volume weighting method for PDE residuals in physics-informed neural networks

VI-PINNs: Variance-involved Physics-informed Neural Networks for Fast and Accurate Prediction of Partial Differential Equations

Cv-Pinn: Efficient Learning of Variational Physics-Informed Neural Network with Domain Decomposition

Meshfree Variational Physics Informed Neural Networks (MF-VPINN): an adaptive training strategy

Recipes for when physics fails: recovering robust learning of physics informed neural networks

Physics-Informed Neural Networks with Adaptive Localized Artificial Viscosity

Variational Physics-informed Neural Operator (VINO) for Solving Partial Differential Equations

VC-PINN: Variable coefficient physics-informed neural network for forward and inverse problems of PDEs with variable coefficient

FastVPINNs: Tensor-Driven Acceleration of VPINNs for Complex Geometries

An efficient hp-Variational PINNs framework for incompressible Navier-Stokes equations

Optimizing Variational Physics-Informed Neural Networks Using Least Squares

Variational Inverting Network for Statistical Inverse Problems of Partial Differential Equations

Variationally Correct Neural Residual Regression for Parametric PDEs: On the Viability of Controlled Accuracy

Bias-Variance Trade-off in Physics-Informed Neural Networks with Randomized Smoothing for High-Dimensional PDEs

Robust Regression with Highly Corrupted Data via Physics Informed Neural Networks