Abstract:In this work, we generalize the reaction-diffusion equation in statistical physics, Schrödinger equation in quantum mechanics, Helmholtz equation in paraxial optics into the neural partial differential equations (NPDE), which can be considered as the fundamental equations in the field of artificial intelligence research. We take finite difference method to discretize NPDE for finding numerical solution, and the basic building blocks of deep neural network architecture, including multi-layer perceptron, convolutional neural network and recurrent neural networks, are generated. The learning strategies, such as Adaptive moment estimation, L-BFGS, pseudoinverse learning algorithms and partial differential equation constrained optimization, are also presented. We believe it is of significance that presented clear physical image of interpretable deep neural networks, which makes it be possible for applying to analog computing device design, and pave the road to physical artificial intelligence.

What problem does this paper attempt to address?

The core problem that this paper attempts to solve is to establish a set of basic theoretical frameworks for the field of artificial intelligence (AI), especially by introducing Neural Partial Differential Equations (NPDEs) to describe and explain the architecture evolution of deep neural networks. Specifically: 1. **Establishing the basic equations of AI**: - The authors propose to generalize the reaction - diffusion equation in statistical physics, the Schrödinger equation in quantum mechanics, and the Helmholtz equation in paraxial optics to Neural Partial Differential Equations (NPDE), which are regarded as the basic equations in the AI research field. 2. **Generating neural network architectures**: - By discretizing NPDE through the Finite Difference Method (FDM), the basic building blocks of deep neural networks are generated, including Multilayer Perceptron (MLP), Convolutional Neural Network (CNN), and Recurrent Neural Network (RNN). This provides a new perspective for understanding neural network architectures. 3. **Optimizing learning strategies**: - Multiple learning strategies have been developed, such as Adaptive Moment Estimation (Adam), L - BFGS, the Pseudo - Inverse Learning Algorithm, and Partial Differential Equation - Constrained Optimization, etc., to improve the training effect of neural networks. 4. **Promoting the development of Physical Artificial Intelligence**: - This work aims to lay a theoretical foundation for the development of Physical Artificial Intelligence (PAI), enabling AI systems to be better applied to analog computing device design and paving the way for the realization of Physical Artificial Intelligence. Through these efforts, the authors hope to explain the working mechanism of deep neural networks from a physical perspective, thereby providing a clearer understanding and higher application possibilities for AI research. ### Formula summary - **Reaction - diffusion equation**: \[ \frac{\partial \Psi}{\partial t}=\nabla\cdot(D(\Psi)\nabla\Psi)+\nabla\cdot V(\Psi)+R(\Psi) \] where \(D(\Psi)\) is the diffusion matrix, \(V(\Psi)\) is the convection vector, and \(R(\Psi)\) is the reaction vector. - **Schrödinger equation**: \[ i\hbar\frac{\partial \psi}{\partial t}=-\frac{\hbar^{2}}{2m}\nabla^{2}\psi + V(x)\psi \] - **Helmholtz equation**: \[ \nabla^{2}u + k^{2}u = 0 \] - **General form of NPDE**: \[ \frac{\partial \Psi}{\partial t}=F[x,t,\nabla,\nabla^{2},D(\Psi),V(\Psi),R(\Psi),\Psi] \] - **Elliptic operator**: \[ O_{\mathcal{L}}\Psi=-\nabla\cdot(D(\Psi)\nabla\Psi)+V(\Psi)\nabla\Psi+R(\Psi) \] These formulas show how to describe and generate neural network architectures through NPDE, thereby providing a solid theoretical foundation for AI research.

Partial Differential Equations is All You Need for Generating Neural Architectures -- A Theory for Physical Artificial Intelligence Systems

An Analysis of Physics-Informed Neural Networks

Deep Neural Networks Motivated by Partial Differential Equations

On the Compatibility between Neural Networks and Partial Differential Equations for Physics-informed Learning

Nonlinear Schrödinger Network

What You See is Not What You Get: Neural Partial Differential Equations and The Illusion of Learning

Three ways to solve partial differential equations with neural networks — A review

Solving Partial Differential Equations Using Deep Learning and Physical Constraints

Solving Partial Differential Equations Using Artificial Neural Networks

An Overview on Machine Learning Methods for Partial Differential Equations: from Physics Informed Neural Networks to Deep Operator Learning

Connections Between Numerical Algorithms for PDEs and Neural Networks

Partial Differential Equations Meet Deep Neural Networks: A Survey

Two-scale Neural Networks for Partial Differential Equations with Small Parameters

On Neural Differential Equations

An Axiomatized PDE Model of Deep Neural Networks.

Deep-learning of parametric partial differential equations from sparse and noisy data

Partial‐differential‐algebraic equations of nonlinear dynamics by physics‐informed neural‐network: (I) Operator splitting and framework assessment

Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations

Learning Interactions in Reaction Diffusion Equations by Neural Networks

Discovering Physics-Informed Neural Networks Model for Solving Partial Differential Equations through Evolutionary Computation

Beyond Finite Layer Neural Networks: Bridging Deep Architectures and Numerical Differential Equations