Abstract:In this paper, we study the partial differential equation models of neural networks. Neural network can be viewed as a map from a simple base model to a complicate function. Based on solid analysis, we show that this map can be formulated by a convection-diffusion equation. This theoretically certified framework gives mathematical foundation and more understanding of neural networks. Moreover, based on the convection-diffusion equation model, we design a novel network structure, which incorporates diffusion mechanism into network architecture. Extensive experiments on both benchmark datasets and real-world applications validate the performance of the proposed model.

What problem does this paper attempt to address?

The paper primarily aims to address the following issues: 1. **Theoretical Framework Construction**: The paper attempts to establish a theoretically rigorous framework to describe the relationship between neural networks (especially Residual Networks, ResNets) and partial differential equations (particularly convection-diffusion equations). Through this framework, the authors hope to provide a further understanding of the mathematical foundations of neural networks. 2. **New Network Structure Design**: Based on the aforementioned theoretical framework, the paper designs a new network structure—the Convection-Diffusion Network (COIN). This network structure introduces a diffusion mechanism on the basis of traditional residual networks. This new structure aims to improve network performance and validate its effectiveness. 3. **Application Validation**: The paper validates the effectiveness of the proposed theoretical framework and new network structure through a series of experiments. Specifically, the experiments include: - Graph Node Classification Task: Validating the performance of COIN in network node classification tasks on benchmark datasets such as Cora, Citeseer, and Pubmed. - Few-Shot Learning Task: Evaluating the performance of COIN in few-shot learning tasks on the miniImageNet, tiered ImageNet, and CUB datasets. - COVID-19 Case Prediction: Investigating the ability of COIN to predict the spread of the pandemic using the England COVID-19 dataset, especially in handling missing data. In summary, the main goal of this paper is to establish a theoretical framework linking neural networks and partial differential equations, design a new network structure COIN based on this framework, and validate the effectiveness and superiority of this structure in different tasks.

Convection-Diffusion Equation: A Theoretically Certified Framework for Neural Networks

An Axiomatized PDE Model of Deep Neural Networks.

Diffusion Mechanism in Residual Neural Network: Theory and Applications

A Flow Model of Neural Networks

Rigorous Justification of the Fokker-Planck Equations of Neural Networks Based on an Iteration Perspective

A deep neural network approach on solving the linear transport model under diffusive scaling

Convolution-Based Model-Solving Method for Three-Dimensional, Unsteady, Partial Differential Equations

PDE Models for Deep Neural Networks: Learning Theory, Calculus of Variations and Optimal Control

Capturing the Diffusive Behavior of the Multiscale Linear Transport Equations by Asymptotic-Preserving Convolutional DeepONets

Neural Message Passing Induced by Energy-Constrained Diffusion

Solving the linear transport equation by a deep neural network approach

Beyond Finite Layer Neural Networks: Bridging Deep Architectures and Numerical Differential Equations

HcNet: Image Modeling with Heat Conduction Equation

Learning the flux and diffusion function for degenerate convection-diffusion equations using different types of observations

Partial Differential Equations is All You Need for Generating Neural Architectures -- A Theory for Physical Artificial Intelligence Systems

The Neural Network Approach to Inverse Problems in Differential Equations

Transformed Physics-Informed Neural Networks for The Convection-Diffusion Equation

On Theory-training Neural Networks to Infer the Solution of Highly Coupled Differential Equations

Intelligence Algorithm Research Of Convection Diffusion Equation And Numerical Simulation

Recent Advances on Dynamical Behaviors of Coupled Neural Networks With and Without Reaction–Diffusion Terms

An Unconstrained Formulation of Some Constrained Partial Differential Equations and its Application to Finite Neuron Methods