Abstract:This paper presents a deep learning feedforward neural network framework as a numerical tool to approximate the solutions to singular nonlinear ordinary differential equations arising in physiology. Artificial neural network (ANN) models are best suited for optimization problems. By casting the solution finding algorithm into a weighted-residual type minimization procedure, the ANN models are shown to be capable of approximating the solution to varieties of equations: ordinary differential equations, partial differential equations and integral equations. In this paper, we utilize a mean-square type loss function in an unsupervised learning network—without actually requiring exact or approximate solution for training—to find the approximate solution to varieties singular differential equations. The total loss function includes the strong form differential equation along with initial/boundary condition terms. During the training phase, our network is trained using a linear interpolation of the small data set, then it adjusts itself to approximate the curvature so that the total loss value is small. The fine-tuning of network’s hyperparameters, which are the weights and bias, is done through the backpropogation step. The efficiency of the proposed method is demonstrated by approximating the solutions to several singular differential equations. We report that the relative error in all the problems are of the order of 10-4$$10^{-4}$$ to 10-7$$10^{-7}$$. The entire computational framework is developed using Python programming platform along with Tensorflow and Keras libraries. Our study has several advantages: first, the entire neural network architecture is easy to implement; second, there is no need for exact solution during the training phase; third, only few data points and a linear interpolation are needed for training and then the network adjusts itself to match the curvature. Furthermore, our study presents a comprehensive comparative examination to settle several issues regarding the choice of various parameters in the network.

A Single-Layer Asymmetric RNN: Potential Low Hardware Complexity Linear Equation Solver

Analog/digital circuit simplification for Hopfield neural network

Matrix Eigenvalue Solver Based on Reconfigurable Photonic Neural Network

A Fast Algorithm to Simulate Nonlinear Resistive Networks

Accurate Mapping of RNNs on Neuromorphic Hardware with Adaptive Spiking Neurons

A Low-Cost Reconfigurable Nonlinear Core for Embedded DNN Applications

Mitigating Asymmetric Nonlinear Weight Update Effects in Hardware Neural Network based on Analog Resistive Synapse

A low cost neuromorphic learning engine based on a high performance supervised SNN learning algorithm

Reconfigurable Multifunction Computing Unit Using an Universal Piecewise Linear Method

Distributed algorithms of solving linear matrix equations via double-layered networks

A One-Layer Dual Recurrent Neural Network with a Heaviside Step Activation Function for Linear Programming with Its Linear Assignment Application

HANDWRITTEN MATHEMATICAL EQUATION SOLVER

Solving Falkner-Skan type equations via Legendre and Chebyshev Neural Blocks

A Compact and Configurable Long Short-Term Memory Neural Network Hardware Architecture.

Machine learning without a processor: Emergent learning in a nonlinear analog network

A Deep Learning Neural Network Framework for Solving Singular Nonlinear Ordinary Differential Equations

Hardware Realization of Building Blocks for Artificial Neural Networks

In-Hardware Learning of Multilayer Spiking Neural Networks on a Neuromorphic Processor

A Software-equivalent SNN Hardware using RRAM-array for Asynchronous Real-time Learning

Neural Network Approach for Solving Nonsingular Multi-Linear Tensor Systems

A Low-Cost Hardware-Friendly Spiking Neural Network Based on Binary MRAM Synapses, Accelerated Using In-Memory Computing