Abstract:Robust nonlinear regression frequently arises in data analysis that is affected by outliers in various application fields such as system identification, signal processing, and machine learning. However, it is still quite challenge to design an efficient algorithm for such problems due to the nonlinearity and nonsmoothness. Previous researches usually ignore the underlying structure presenting in the such nonlinear regression models, where the variables can be partitioned into a linear part and a nonlinear part. Inspired by the high efficiency of variable projection algorithm for solving separable nonlinear least squares problems, in this article, we develop a robust variable projection (RoVP) method for the parameter estimation of separable nonlinear regression problem with L-1 norm loss. The proposed algorithm eliminates the linear parameters by solving a linear programming subproblem, resulting in a reduced problem that only involves nonlinear parameters. More importantly, we derive the Jacobian matrix of the reduced objective function, which tackles the coupling between the linear parameters and nonlinear parameters. Furthermore, we observed an intriguing phenomenon in the landscape of the original separable nonlinear objective function, where some narrow valleys frequently exist. The RoVP strategy can effectively diminish the likelihood of the algorithm getting stuck in these valleys and accelerate its convergence. Numerical experiments confirm the effectiveness and robustness of the proposed algorithm.

An Efficient Algorithm for Ill-Conditioned Separable Nonlinear Least Squares

An efficient variable projection formulation for separable nonlinear least squares problems.

A New Algorithm For Unconstrained Optimization Problem With The Form Of Sum Of Squares Minimization

Efficient Parameters Estimation Method for the Separable Nonlinear Least Squares Problem

Insights Into Algorithms for Separable Nonlinear Least Squares Problems

On Some Separated Algorithms for Separable Nonlinear Least Squares Problems

An Iterative Implementation of Variable Projection for Separable Nonlinear Optimization Problems

A Regularized Variable Projection Algorithm for Separable Nonlinear Least-Squares Problems.

Robust Variable Projection Algorithm for the Identification of Separable Nonlinear Models

Separable nonlinear least squares: the variable projection method and its applications

Modified Gram–Schmidt Method-Based Variable Projection Algorithm for Separable Nonlinear Models

Separable Algorithms for Matrix Factorization with Presence of Missing Data

Regularization Methods for Separable Nonlinear Models

Variable projection algorithms with sparse constraint for separable nonlinear models

Online Identification of Nonlinear Systems With Separable Structure

Proving Nonsingularity of Coefficient Matrix in Least Squares Normal Equation of Non-Linear Semiparametric Model

Recursive Variable Projection Algorithm for a Class of Separable Nonlinear Models

An Efficient Hessian Based Algorithm for Singly Linearly and Box Constrained Least Squares Regression.

Sequential Quadratic Programming Method for Nonlinear Least Squares Estimation and Its Application

A Preconditioner based on Non-uniform Row Sampling for Linear Least Squares Problems

Sparse LSSVM Algorithm Based on QR Factorization