Abstract:Modeling and simulation of complex non-linear systems are essential in physics, engineering, and signal processing. Neural networks are widely regarded for such tasks due to their ability to learn complex representations from data. Training deep neural networks traditionally requires large amounts of data, which may not always be readily available for such systems. Contrarily, there is a large amount of domain knowledge in the form of mathematical models for the physics/behavior of such systems. A new class of neural networks called Physics-Informed Neural Networks (PINNs) has gained much attention recently as a paradigm for combining physics into neural networks. They have become a powerful tool for solving forward and inverse problems involving differential equations. A general framework of a PINN consists of a multi-layer perceptron that learns the solution of the partial differential equation (PDE) along with its boundary/initial conditions by minimizing a multi-objective loss function. This is formed by the sum of individual loss terms that penalize the output at different collocation points based on the differential equation and initial and boundary conditions. However, multiple loss terms arising from PDE residual and boundary conditions in PINNs pose a challenge in optimizing the overall loss function. This often leads to training failures and inaccurate results. We propose advanced gradient statistics-based weighting schemes for PINNs to address this challenge. These schemes utilize backpropagated gradient statistics of individual loss terms to appropriately scale and assign weights to each term, ensuring balanced training and meaningful solutions. In addition to the existing gradient statistics-based weighting schemes, we introduce kurtosis-standard deviation-based and combined mean and standard deviation-based schemes for approximating solutions of PDEs using PINNs. We provide a qualitative and quantitative comparison of these weighting schemes on 2D Poisson's and Klein-Gordon's equations, highlighting their effectiveness in improving PINN performance.

Physics-Guided State-Space Model Augmentation Using Weighted Regularized Neural Networks

Physics-guided Neural Networks (PGNN): An Application in Lake Temperature Modeling

Physics-guided generative adversarial network for probabilistic structural system identification

Physics-guided generative adversarial network to learn physical models

Physics-guided Deep Markov Models for Learning Nonlinear Dynamical Systems with Uncertainty

Adaptive Physics-Guided Neural Network

Physics-guided machine learning from simulated data with different physical parameters

Physics-guided neural networks for feedforward control with input-to-state-stability guarantees

Physics-guided neural networks for feedforward control with input-to-state stability guarantees

Physics-Driven Regularization of Deep Neural Networks for Enhanced Engineering Design and Analysis

State Space Model-Based Runge–Kutta Gated Recurrent Unit Networks for Structural Response Prediction

Physics Inspired Approaches To Understanding Gaussian Processes

Physics-Constraint Variational Neural Network for Wear State Assessment of External Gear Pump

Physics-guided Data Augmentation for Learning the Solution Operator of Linear Differential Equations

PIAug -- Physics Informed Augmentation for Learning Vehicle Dynamics for Off-Road Navigation

Recipes for when physics fails: recovering robust learning of physics informed neural networks

Physics-guided Active Sample Reweighting for Urban Flow Prediction

Gradient Statistics-Based Multi-Objective Optimization in Physics-Informed Neural Networks

PhyMPGN: Physics-encoded Message Passing Graph Network for spatiotemporal PDE systems

Physical Rule-Guided Convolutional Neural Network

Physics-Guided Architecture (PGA) of Neural Networks for Quantifying Uncertainty in Lake Temperature Modeling