Abstract:Sparse optimization involving the L0-norm function as the regularization in objective function has a wide application in many fields. In this paper, we propose a projected neural network modeled by a differential equation to solve a class of these optimization problems, in which the objective function is the sum of a nonsmooth convex loss function and the regularization defined by the L0-norm function. This optimization problem is not only nonconvex, but also discontinuous. To simplify the structure of the proposed network and let it own better convergence properties, we use the smoothing method, where the new constructed smoothing function for the regularization term plays a key role. We prove that the solution to the proposed network is globally existent and unique, and any accumulation point of it is a critical point of the continuous relaxation model. Except for a special case, which can be easily justified, any critical point is a local minimizer of the considered sparse optimization problem. It is an interesting thing that all critical points own a promising lower bound property, which is satisfied by all global minimizers of the considered problem, but is not by all local minimizers. Finally, we use some numerical experiments to illustrate the efficiency and good performance of the proposed method for solving this class of sparse optimization problems, which include the most widely used models in feature selection of classification learning.

An Improved Echo State Network via L 1-Norm Regularization

L1/2 Norm Regularized Echo State Network for Chaotic Time Series Prediction.

Echo State Network Based on L0 Norm Regularization for Chaotic Time Series Prediction.

Design of Echo State Network with Multi-Objective Optimization Algorithm and L1 Regularization

Adaptive lasso echo state network based on modified Bayesian information criterion for nonlinear system modeling

An Echo State Network Based on Levenberg-Marquardt Algorithm

An Effective Online Sparse Learning Algorithm for Echo State Networks

An Norm 1 Regularization Term ELM Algorithm Based on Surrogate Function and Bayesian Framework

A Stability Criterion for Discrete-Time Fractional-Order Echo State Network and Its Application

Design of Sparse Bayesian Echo State Network for Time Series Prediction

Hybrid Regularized Echo State Network for Multivariate Chaotic Time Series Prediction

Neural Network for a Class of Sparse Optimization with L0-regularization.

Online Sequential Echo State Network with Sparse RLS Algorithm for Time Series Prediction.

Regularized Taylor Echo State Networks for Predictive Control of Partially Observed Systems.

Adaptive Levenberg-Marquardt Algorithm Based Echo State Network for Chaotic Time Series Prediction

A Simple Neural Network for Sparse Optimization with $l_1$ Regularization

Design of Incremental Echo State Network Using Leave-One-Out Cross-Validation.

Multi-objective Sparse Echo State Network

A New Neural Network for L(1)-Norm Programming

LM Algorithm in Echo State Network for Chaotic Time Series Prediction

Long-Short Term Echo State Network for Time Series Prediction