Abstract:Sparse optimization involving the L0-norm function as the regularization in objective function has a wide application in many fields. In this paper, we propose a projected neural network modeled by a differential equation to solve a class of these optimization problems, in which the objective function is the sum of a nonsmooth convex loss function and the regularization defined by the L0-norm function. This optimization problem is not only nonconvex, but also discontinuous. To simplify the structure of the proposed network and let it own better convergence properties, we use the smoothing method, where the new constructed smoothing function for the regularization term plays a key role. We prove that the solution to the proposed network is globally existent and unique, and any accumulation point of it is a critical point of the continuous relaxation model. Except for a special case, which can be easily justified, any critical point is a local minimizer of the considered sparse optimization problem. It is an interesting thing that all critical points own a promising lower bound property, which is satisfied by all global minimizers of the considered problem, but is not by all local minimizers. Finally, we use some numerical experiments to illustrate the efficiency and good performance of the proposed method for solving this class of sparse optimization problems, which include the most widely used models in feature selection of classification learning.

A new Sigma-Pi-Sigma neural network based on $ L_1 $ and $ L_2 $ regularization and applications

Convergence of Batch Gradient Learning Algorithm with Smoothing L1/2 Regularization for Sigma–Pi–Sigma Neural Networks

Convergence of online gradient method for pi-sigma neural networks

A Modified Higher-Order Feed Forward Neural Network with Smoothing Regularization

Convergence of online gradient methods for Pi-Sigma neural network with a penalty term

A Modified Sigma-Pi-Sigma Neural Network with Adaptive Choice of Multinomials.

Neural Network for a Class of Sparse Optimization with L0-regularization.

Convergence Analysis of Batch Gradient Algorithm for Three Classes of Sigma-Pi Neural Networks

A New Neural Network Model And Its Application

Training Pi-Sigma Network by Online Gradient Algorithm with Penalty for Small Weight Update.

Convergence of gradient descent algorithm for Pi-sigma neural networks

A New Conjugate Gradient Method with Smoothing L_1/2 Regularization Based on a Modified Secant Equation for Training Neural Networks

A New Conjugate Gradient Method with Smoothing $$L_{1/2} $$L1/2 Regularization Based on a Modified Secant Equation for Training Neural Networks

A Simple Neural Network for Sparse Optimization with $l_1$ Regularization

A New Neural Network for L(1)-Norm Programming

Batch gradient method with smoothing L1/2 regularization for training of feedforward neural networks.

A Modified Gradient Learning Algorithm with Smoothing L-1/2 Regularization for Takagi-Sugeno Fuzzy Models

Convergence of online gradient method for feedforward neural networks with smoothing L 1/2 regularization penalty

An Improved Echo State Network via L 1-Norm Regularization

New Sigma-Pi Functional Networks Model

Application of Pi-Sigma network in underwater acoustic object classification