Abstract:Physics-Informed Neural Networks (PINNs) have emerged as a powerful tool for solving partial differential equations~(PDEs) in various scientific and engineering domains. However, traditional PINN architectures typically rely on large, fully connected multilayer perceptrons~(MLPs), lacking the sparsity and modularity inherent in many traditional numerical solvers. An unsolved and critical question for PINN is: What is the minimum PINN complexity regarding nodes, layers, and connections needed to provide acceptable performance? To address this question, this study investigates a novel approach by merging established PINN methodologies with brain-inspired neural network techniques. We use Brain-Inspired Modular Training~(BIMT), leveraging concepts such as locality, sparsity, and modularity inspired by the organization of the brain. With brain-inspired PINN, we demonstrate the evolution of PINN architectures from large, fully connected structures to bare-minimum, compact MLP architectures, often consisting of a few neural units!

What problem does this paper attempt to address?

The problem this paper attempts to address is: how to design a Physics-Informed Neural Network (PINN) architecture that minimizes the number of neural units, layers, and connections, in order to improve computational efficiency and resource utilization while ensuring accuracy. Specifically, traditional PINN architectures typically rely on large, fully connected Multi-Layer Perceptrons (MLPs), lacking the sparsity and modularity found in traditional numerical solvers. Therefore, this study explores the evolution from large fully connected structures to minimized compact MLP architectures by introducing brain-inspired neural network techniques (such as locality, sparsity, and modularity). The main objectives of the study include: 1. **Determining the minimized PINN architecture**: The study explores minimized PINN architectures capable of effectively solving Partial Differential Equations (PDEs) through the Brain-Inspired Module Training (BIMT) method. For example, for simple differential equations like the logistic equation, only 1 hidden layer neuron is needed to solve it. 2. **Spectral bias phenomenon**: The study demonstrates that minimized architectures require more neurons to solve high-frequency problems, while fewer neurons are needed for low-frequency problems. This indicates that spectral bias not only affects the convergence rate but also influences the design of the PINN architecture. 3. **Modular PINN architecture**: Through BIMT training, the study derives fundamental PINN building modules, which are similar to common deep learning building blocks such as convolutional kernels and attention mechanisms. These modules can be combined into larger modular PINN architectures, thereby reducing computational and memory resource consumption while maintaining performance. In summary, this paper aims to design minimized and efficient PINN architectures through brain-inspired techniques to address the optimization of computational resources in PDE solving.

Brain-Inspired Physics-Informed Neural Networks: Bare-Minimum Neural Architectures for PDE Solvers

A Spatial-Temporally Adaptive PINN Framework for 3D Bi-Ventricular Electrophysiological Simulations and Parameter Inference

The Old and the New: Can Physics-Informed Deep-Learning Replace Traditional Linear Solvers?

Physics Informed Neural Networks (PINNs) as intelligent computing technique for solving partial differential equations: Limitation and Future prospects

Architectural Strategies for the optimization of Physics-Informed Neural Networks

SetPINNs: Set-based Physics-informed Neural Networks

Scientific Machine Learning Through Physics–Informed Neural Networks: Where we are and What's Next

Multilevel domain decomposition-based architectures for physics-informed neural networks

Machine Learning Through Physics–Informed Neural Networks: Progress and Challenges

NAS-PINN: Neural architecture search-guided physics-informed neural network for solving PDEs

Physics-informed attention-based neural network for solving non-linear partial differential equations

Convergence and error analysis of PINNs

NeuralPDE: Automating Physics-Informed Neural Networks (PINNs) with Error Approximations

Densely Multiplied Physics Informed Neural Networks

On the Compatibility between Neural Networks and Partial Differential Equations for Physics-informed Learning

Element-wise Multiplication Based Deeper Physics-Informed Neural Networks

Binary structured physics-informed neural networks for solving equations with rapidly changing solutions

Finite-PINN: A Physics-Informed Neural Network Architecture for Solving Solid Mechanics Problems with General Geometries

PINNacle: A Comprehensive Benchmark of Physics-Informed Neural Networks for Solving PDEs

A novel discretized physics-informed neural network model applied to the Navier-Stokes equations

An Investigation of Physics Informed Neural Networks to solve the Poisson-Boltzmann Equation in Molecular Electrostatics