Abstract:Variational inequality (VI) can be viewed as a natural framework for unifying the treatment of equilibrium problems, and hence has applications across many disciplines. In addition, many typical problems are closely related to VI, including general VI (GVI), complementarity problem (CP), generalized CP (GCP) and optimization problem (OP). During the past two decades, numerous recurrent neural networks (RNNs) have been proposed for solving VIs and related problems. However, first, the theories of many emerging RNNs have not been well founded yet; and their capabilities have been underestimated. Second, these RNNs have limitations in handling some types of problems. Third, it is certainly not true that these RNNs are best choices for solving all problems, and new network models with more favorable characteristics could be devised for solving specific problems. In the research, the above issues are extensively explored from dynamic system perspective, which leads to the following major contributions. On one hand, many new capabilities of some existing RNNs have been revealed for solving VIs and related problems. On the other hand, several new RNNs have been invented for solving some types of these problems. The contributions are established on the following facts. First, two existing RNNs, called TLPNN and PNN, are found to be capable of solving pseudomonotone VIs and related problems with simple bound constraints. Second, many more stability results are revealed for an existing RNN, called GPNN, for solving GVIs with simple bound constraints, and it is then extended to solve linear VIs (LVIs) and generalized linear VIs (GLVIs) with polyhedron constraints. Third, a new RNN, called IDNN, is proposed for solving a special class of quadratic programming problems which features lower structural complexity compared with existing RNNs. Fourth, some local convergence results of an existing RNN, called EPNN, for nonconvex optimization are obtained, and two variants of the network by incorporating two augmented Lagrangian function techniques are proposed for seeking Karush-Kuhn-Tucker (KKT) points, especially local optima, of the problems.

Analogue Recurrent Neural Network for Linear Algebraic Equation Solving

A fast algorithm for solving large scale nonlinear optimization problems using RNN

Design and Analysis of a Novel Integral Recurrent Neural Network for Solving Time-Varying Sylvester Equation

Recurrent Neural Networks for Recursive Identification of Nonlinear Dynamic Process

An Adaptive Gradient Neural Network to Solve Dynamic Linear Matrix Equations

Convergence analysis of discrete time recurrent neural networks for linear variational inequality problem

Stability analysis of a class of discrete-time recurrent neural networks: An LMI approach

An efficient zeroing neural network for solving time-varying nonlinear equations

A finite-time recurrent neural network for solving online time-varying Sylvester matrix equation based on a new evolution formula

An Improved Finite Time Convergence Recurrent Neural Network with Application to Time-Varying Linear Complex Matrix Equation Solution

Robust Exponential Stability Analysis of a Larger Class of Discrete-Time Recurrent Neural Networks

Learning nonlinear integral operators via Recurrent Neural Networks and its application in solving Integro-Differential Equations

New Approach for Robust Stability Analysis of Delayed Recurrent Neural Networks

Solving variational inequalities and related problems using recurrent neural networks

Linear RNNs Provably Learn Linear Dynamical Systems

A Novel Linear Recurrent Neural Network for Multivariable System Identification

A Novel Recurrent Neural Network-Based Ultra-Fast, Robust, and Scalable Solver for Inverting a "Time-Varying Matrix"

Recurrent neural networks for dynamical systems: Applications to ordinary differential equations, collective motion, and hydrological modeling

Linear RNNs Provably Learn Linear Dynamic Systems

Stability Analysis of Recurrent Multilayer Perceptrons:Lmi Approach

A Deep Learning Neural Network Framework for Solving Singular Nonlinear Ordinary Differential Equations