Abstract:The field of neural differential equations has witnessed significant growth in the past decade, with a notable increase in published literature. This approach offers several advantages, including the ability to model complex, nonlinear systems and the potential for extrapolation beyond the observed data. There is a need of systematic overview in implementing the use of neural networks in solving differential equations and their special cases. This work is a discussion on architectures of neural networks used in NDEs, training methodologies and applications across various domains. The current work reviews and categorizes earlier study and gives guidelines to mathematicians, researchers in computer science and engineering. Also, it includes advantages of NDEs and some current challenges and research questions in the field of NDEs. Introduction: The field of neural differential equations has witnessed significant growth in the past decade, with a notable increase in published literature. This approach offers several advantages, including the ability to model complex, nonlinear systems and the potential for extrapolation beyond the observed data. There is a need of systematic overview in implementing the use of neural networks in solving differential equations and their special cases. This work is a discussion on architectures of neural networks used in NDEs, training methodologies and applications across various domains. The current work reviews and categorizes earlier study and gives guidelines to mathematicians, researchers in computer science and engineering. Also, it includes advantages of NDEs and some current challenges and research questions in the field of NDEs. Conclusions: The authors have discussed the observations of ODEs, Neural network and the literature review of NDEs. The overall framework of neural networks for solving differential equations is also covered in this paper. This paper also includes general features of neural algorithms for solving differential equations. Neural networks may approximate any continuous system and stabilize the training process with the help of suitable activation functions. It allows us to solve problems more quickly without wasting processing power or memory. The parallel nature of the network helps to minimize the complexity of the problems, as demonstrated by a comparison with the exact solution, along with modeling various dynamical processes. Additionally, one can solve fractional differential equations using Neural network and many more applications based on it.

Neural Fractional Differential Equations

Optimising the Order of the Fractional Derivative

deepFDEnet: A Novel Neural Network Architecture for Solving Fractional Differential Equations

Solving Fractional Order Differential Equations by Using Fractional Radial Basis Function Neural Network

A Data-Driven Framework for Discovering Fractional Differential Equations in Complex Systems

An Artificial Neural Network Approach for Solving Space Fractional Differential Equations

Fibonacci Neural Network Approach for Numerical Solutions of Fractional Order Differential Equations

Application of Intelligent Paradigm through Neural Networks for Numerical Solution of Multiorder Fractional Differential Equations

Coupling Graph Neural Networks with Fractional Order Continuous Dynamics: A Robustness Study

A new modern scheme for solving fractal-fractional differential equations based on deep feedforward neural network with multiple hidden layer

On Neural Differential Equations

Finite Difference Neural Networks: Fast Prediction of Partial Differential Equations

Neural Rough Differential Equations for Long Time Series

Fractional Deep Neural Network via Constrained Optimization

Neural Differential Equations: A Comprehensive Review and Applications

Fractional SDE-Net: Generation of Time Series Data with Long-term Memory

A novel optimization-based physics-informed neural network scheme for solving fractional differential equations

PMNN:Physical Model-driven Neural Network for solving time-fractional differential equations

Neural Piecewise-Constant Delay Differential Equations

Fourier Neural Differential Equations for learning Quantum Field Theories

Neural Differential Algebraic Equations