Abstract:1 Introduction Typically, models of neural networks are divided into two categories in terms of signal transmission manner: feed-forward neural networks and recurrent neural networks. They are built up using different frameworks, which give rise to different fields of applications. Feed-forward neural network (FNN), also referred to as multilayer percep-trons (MLPs), has drawn great interests over the last two decades for its distinction as a universal function approximator As an important intelligent computation method, FNN has been applied to a wide range of applications, including curve fitting, pattern classification and nonlinear system identification and so on (Vemuri, 1995). FNN features a supervised training with a highly popular algorithm known as the error back-propagation algorithm. In the standard back-propagation (SBP) algorithm, the learning of a FNN is composed of two passes: in the forward pass, the input signal propagates through the network in a forward direction, on a layer-by-layer basis with the weights fixed; in the backward pass, the error signal is propagated in a backward manner. The weights are adjusted based on an error-correction rule. Although it has been successfully used in many real world applications, SBP suffers from two infamous shortcomings, i.e., slow learning speed and sensitivity to parameters. Many iterations are required to train small networks, even for a simple problem. The sensitivity to learning parameters, initial states and perturbations was analyzed in (Yeung and Sun, 2002). Behind such drawbacks the learning rate plays a key role in affecting the learning performance and it has to be chosen carefully. If the learning rate is large, the network may exhibit chaotic

Modelling of the Relaxation Least Squares-Based Neural Networks and Its Application

Application study of a kind of novel neural networks models

A Neural Network Model For General Minimax Problem

Neural Network Solution for General Nonlinear Optimization Problems

A discrete-time neural network for optimization problems with hybrid constraints.

A New Neural Network Model And Its Application

Neural Networks: Computational Models and Applications - ReadingSample

Optimization Over Trained Neural Networks: Taking a Relaxing Walk

Two Recurrent Neural Networks With Reduced Model Complexity for Constrained l1-Norm Optimization

Capturing Shock Waves by Relaxation Neural Networks

Representation and Regression Problems in Neural Networks: Relaxation, Generalization, and Numerics

Nonlinear System Modeling Using Multiple Neural Networks

New zeroing NN models with nonconvex saturated activation functions in noisy environments for quadratic minimization dynamics and control

A Structure-Guided Gauss-Newton Method for Shallow ReLU Neural Network

Generalized Levenberg-Marquardt Neural Nets for Minimization of Quasiconvex Scalar Functions

Efficient SGD Neural Network Training via Sublinear Activated Neuron Identification

Tightened Convex Relaxations for Neural Network Robustness Certification

Tight Neural Network Verification via Semidefinite Relaxations and Linear Reformulations

An Artificial Neural-Network-Based Approach to Software Reliability Assessment

Gauss Newton method for solving variational problems of PDEs with neural network discretizaitons