Abstract:Modeling and simulation of complex non-linear systems are essential in physics, engineering, and signal processing. Neural networks are widely regarded for such tasks due to their ability to learn complex representations from data. Training deep neural networks traditionally requires large amounts of data, which may not always be readily available for such systems. Contrarily, there is a large amount of domain knowledge in the form of mathematical models for the physics/behavior of such systems. A new class of neural networks called Physics-Informed Neural Networks (PINNs) has gained much attention recently as a paradigm for combining physics into neural networks. They have become a powerful tool for solving forward and inverse problems involving differential equations. A general framework of a PINN consists of a multi-layer perceptron that learns the solution of the partial differential equation (PDE) along with its boundary/initial conditions by minimizing a multi-objective loss function. This is formed by the sum of individual loss terms that penalize the output at different collocation points based on the differential equation and initial and boundary conditions. However, multiple loss terms arising from PDE residual and boundary conditions in PINNs pose a challenge in optimizing the overall loss function. This often leads to training failures and inaccurate results. We propose advanced gradient statistics-based weighting schemes for PINNs to address this challenge. These schemes utilize backpropagated gradient statistics of individual loss terms to appropriately scale and assign weights to each term, ensuring balanced training and meaningful solutions. In addition to the existing gradient statistics-based weighting schemes, we introduce kurtosis-standard deviation-based and combined mean and standard deviation-based schemes for approximating solutions of PDEs using PINNs. We provide a qualitative and quantitative comparison of these weighting schemes on 2D Poisson's and Klein-Gordon's equations, highlighting their effectiveness in improving PINN performance.

A novel optimization-based physics-informed neural network scheme for solving fractional differential equations

Physics informed neural network based scheme and its error analysis for ψ-Caputo type fractional differential equations

A Physics Informed Neural Network for Time-Dependent Nonlinear and Higher Order Partial Differential Equations

Physics-informed Neural Networks for Functional Differential Equations: Cylindrical Approximation and Its Convergence Guarantees

A novel discretized physics-informed neural network model applied to the Navier-Stokes equations

Deep neural network methods for solving forward and inverse problems of time fractional diffusion equations with conformable derivative

Optimal Transport Based Refinement of Physics-Informed Neural Networks

An inverse problem of determining the parameters in diffusion equations by using fractional physics-informed neural networks

Bi-orthogonal fPINN: A physics-informed neural network method for solving time-dependent stochastic fractional PDEs

Distributed physics informed neural network for data-efficient solution to partial differential equations

Gradient Statistics-Based Multi-Objective Optimization in Physics-Informed Neural Networks

Laplace-fPINNs: Laplace-based fractional physics-informed neural networks for solving forward and inverse problems of subdiffusion

Physics-Informed Neural Network based inverse framework for time-fractional differential equations for rheology

Accelerating Fractional PINNs using Operational Matrices of Derivative

Efficient Discrete Physics-informed Neural Networks for Addressing Evolutionary Partial Differential Equations

Physics-Informed Kernel Function Neural Networks for Solving Partial Differential Equations

Physics-informed neural networks combined with polynomial interpolation to solve nonlinear partial differential equations

A hybrid physics-informed neural network based multiscale solver as a partial differential equation constrained optimization problem

Physics Informed Neural Networks (PINNs) as intelligent computing technique for solving partial differential equations: Limitation and Future prospects

A novel numerical approach for time-varying impulsive fractional differential equations using theory of functional connections and neural network

Score-fPINN: Fractional Score-Based Physics-Informed Neural Networks for High-Dimensional Fokker-Planck-Levy Equations