Abstract:The NEPv approach has been increasingly used lately for optimization on the Stiefel manifold arising from machine learning. General speaking, the approach first turns the first order optimality condition into a nonlinear eigenvalue problem with eigenvector dependency (NEPv) and then solve the nonlinear problem via some variations of the self-consistent-field (SCF) iteration. The difficulty, however, lies in designing a proper SCF iteration so that a maximizer is found at the end. Currently, each use of the approach is very much individualized, especially in its convergence analysis phase to show that the approach does work or otherwise. Related, the NPDo approach is recently proposed for the sum of coupled traces and it seeks to turn the first order optimality condition into a nonlinear polar decomposition with orthogonal factor dependency (NPDo). In this paper, two unifying frameworks are established, one for each approach. Each framework is built upon a basic assumption, under which globally convergence to a stationary point is guaranteed and during the SCF iterative process that leads to the stationary point, the objective function increases monotonically. Also the notion of atomic function for each approach is proposed, and the atomic functions include commonly used matrix traces of linear and quadratic forms as special ones. It is shown that the basic assumptions of the approaches are satisfied by their respective atomic functions and, more importantly, by convex compositions of their respective atomic functions. Together they provide a large collection of objectives for which either one of approaches or both are guaranteed to work, respectively.

A neural network based on the generalized Fischer–Burmeister function for nonlinear complementarity problems

A neurofuzzy network structure for nonlinear system modeling

Neural Network Solution for General Nonlinear Optimization Problems

A Neural Network Model For General Minimax Problem

A Neural Network Approach for Solving Weighted Nonlinear Complementarity Problems

Improved Smoothing Newton Methods for P0 Nonlinear Complementarity Problems

Superlinear／Quadratic One-step Smoothing Newton Methodf or P0-NCP

Superlinear/Quadratic One–step Smoothing Newton Method for P 0–NCP

A neural network model for non-smooth optimization over a compact convex subset

PFNN: A Penalty-Free Neural Network Method for Solving a Class of Second-Order Boundary-Value Problems on Complex Geometries

Generalized Gradient Projection Neural Networks for Nonsmooth Optimization Problems

Unified Control Liapunov Function Based Design of Neural Networks That Aim at Global Minimization of Nonconvex Functions

An approximate proximal point algorithm for nonlinear complementarity problems

Comparative Study of the CG and HBF ODEs Used in the Global Minimization of Nonconvex Functions

A Constrained Optimization Approach for Lcp

NE/SQP: A robust algorithm for the nonlinear complementarity problem

The Non-Interior Continuation Methods for Solving the P 0 Function Nonlinear Complementarity Problem

A cooperative neural dynamic model for solving general convex nonlinear optimization problems with fuzzy parameters and an application in manufacturing systems

A neuron-wise subspace correction method for the finite neuron method

A Theory of the NEPv Approach for Optimization on the Stiefel Manifold