Abstract:The field of neural differential equations has witnessed significant growth in the past decade, with a notable increase in published literature. This approach offers several advantages, including the ability to model complex, nonlinear systems and the potential for extrapolation beyond the observed data. There is a need of systematic overview in implementing the use of neural networks in solving differential equations and their special cases. This work is a discussion on architectures of neural networks used in NDEs, training methodologies and applications across various domains. The current work reviews and categorizes earlier study and gives guidelines to mathematicians, researchers in computer science and engineering. Also, it includes advantages of NDEs and some current challenges and research questions in the field of NDEs. Introduction: The field of neural differential equations has witnessed significant growth in the past decade, with a notable increase in published literature. This approach offers several advantages, including the ability to model complex, nonlinear systems and the potential for extrapolation beyond the observed data. There is a need of systematic overview in implementing the use of neural networks in solving differential equations and their special cases. This work is a discussion on architectures of neural networks used in NDEs, training methodologies and applications across various domains. The current work reviews and categorizes earlier study and gives guidelines to mathematicians, researchers in computer science and engineering. Also, it includes advantages of NDEs and some current challenges and research questions in the field of NDEs. Conclusions: The authors have discussed the observations of ODEs, Neural network and the literature review of NDEs. The overall framework of neural networks for solving differential equations is also covered in this paper. This paper also includes general features of neural algorithms for solving differential equations. Neural networks may approximate any continuous system and stabilize the training process with the help of suitable activation functions. It allows us to solve problems more quickly without wasting processing power or memory. The parallel nature of the network helps to minimize the complexity of the problems, as demonstrated by a comparison with the exact solution, along with modeling various dynamical processes. Additionally, one can solve fractional differential equations using Neural network and many more applications based on it.

The advance of neural ordinary differential ordinary differential equations

Neural Differential Equations: A Comprehensive Review and Applications

Neural Ordinary Differential Equation based Recurrent Neural Network Model

ODEN: A Framework to Solve Ordinary Differential Equations using Artificial Neural Networks

Neural Ordinary Differential Equations with Envolutionary Weights

Do Residual Neural Networks discretize Neural Ordinary Differential Equations?

Accuracy and Architecture Studies of Residual Neural Network solving Ordinary Differential Equations

New Designed Loss Functions to Solve Ordinary Differential Equations with Artificial Neural Network

On Neural Differential Equations

Neural Generalized Ordinary Differential Equations with Layer-varying Parameters

Constrained Neural Ordinary Differential Equations with Stability Guarantees

Neural Delay Differential Equations: System Reconstruction and Image Classification

Differential Equations for Continuous-Time Deep Learning

Neural Piecewise-Constant Delay Differential Equations

Reachability Analysis of a General Class of Neural Ordinary Differential Equations

Neural Network As a Function Approximator and Its Application in Solving Differential Equations

A Survey on Solving and Discovering Differential Equations Using Deep Neural Networks

Beyond Finite Layer Neural Networks: Bridging Deep Architectures and Numerical Differential Equations

Solving Irregular and Data-enriched Differential Equations using Deep Neural Networks

A Deep Neural Network Framework for Solving Forward and Inverse Problems in Delay Differential Equations

A Deep Learning Neural Network Framework for Solving Singular Nonlinear Ordinary Differential Equations