Abstract:Generalized gradient projection neural network models are proposed to solve nonsmooth convex and nonconvex nonlinear programming problems over a closed convex subset of ℝ n . By using Clarke’s generalized gradient, the neural network modeled by a differential inclusion is developed, and its dynamical behavior and optimization capabilities both for convex and nonconvex problems are rigorously analyzed in the framework of nonsmooth analysis and the differential inclusion theory. First for nonconvex optimization problems, the quasiconvergence results similar to those achieved by previous work are proved. For convex optimization problems, the global convergence results are proved, i.e. any trajectory of the neural network converges to an asymptotically stable equilibrium point, which is an optimal solution of the primal problem whenever it exists. This result remains valid even if the initial condition is chosen outside the feasible set. In addition, an asymptotic control result involving a Tykhonov-like regularization shows that any trajectory of the revised neural network can be forced to converge toward a particular optimal solution of the primal problem. Finally, two simulation optimization algorithms for solving the optimization and control problems are designed respectively, and three typical simulation experiments are illustrated to present the accuracy and efficiency of theoretical convergence results of this paper.

A generalized global convergence theory of projection-type neural networks for optimization

New Critical Analysis on Global Convergence of Recurrent Neural Networks with Projection Mappings

Global Exponential Stability of Discrete-Time Neural Networks for Constrained Quadratic Optimization

On exponential convergence conditions of an extended projection neural network

Projected Neural Network for a Class of Non-Lipschitz Optimization Problems with Linear Constraints.

A Critical Global Convergence Analysis of Recurrent Neural Networks with General Projection Mappings

Generalized Gradient Projection Neural Networks for Nonsmooth Optimization Problems

A Critical Analysis on Global Convergence of Hopfield-type Neural Networks.

O(log T) Projections for Stochastic Optimization of Smooth and Strongly Convex Functions

A discrete-time neural network for optimization problems with hybrid constraints.

Global Convergence Analysis of Deep Linear Networks with A One-neuron Layer

Convergences for Minimax Optimization Problems over Infinite-Dimensional Spaces Towards Stability in Adversarial Training

PDE-constrained Models with Neural Network Terms: Optimization and Global Convergence

A Convergence Theory Towards Practical Over-parameterized Deep Neural Networks

Global Exponential Stability and Global Convergence in Finite Time of Neural Networks with Discontinuous Activations

A Local Convergence Theory for Mildly Over-Parameterized Two-Layer Neural Network

Convergence in Discrete-Time Neural Networks with Specific Performance.

Global Exponential Convergence Analysis of Delayed Neural Networks with Time-Varying Delays

On Convergence of Training Loss Without Reaching Stationary Points

Three Global Exponential Convergence Results of the GPNN for Solving Generalized Linear Variational Inequalities

Solving Pseudomonotone Variational Inequalities and Pseudoconvex Optimization Problems Using the Projection Neural Network.