Abstract:Robust nonlinear regression frequently arises in data analysis that is affected by outliers in various application fields such as system identification, signal processing, and machine learning. However, it is still quite challenge to design an efficient algorithm for such problems due to the nonlinearity and nonsmoothness. Previous researches usually ignore the underlying structure presenting in the such nonlinear regression models, where the variables can be partitioned into a linear part and a nonlinear part. Inspired by the high efficiency of variable projection algorithm for solving separable nonlinear least squares problems, in this article, we develop a robust variable projection (RoVP) method for the parameter estimation of separable nonlinear regression problem with L-1 norm loss. The proposed algorithm eliminates the linear parameters by solving a linear programming subproblem, resulting in a reduced problem that only involves nonlinear parameters. More importantly, we derive the Jacobian matrix of the reduced objective function, which tackles the coupling between the linear parameters and nonlinear parameters. Furthermore, we observed an intriguing phenomenon in the landscape of the original separable nonlinear objective function, where some narrow valleys frequently exist. The RoVP strategy can effectively diminish the likelihood of the algorithm getting stuck in these valleys and accelerate its convergence. Numerical experiments confirm the effectiveness and robustness of the proposed algorithm.

Recursive Variable Projection Algorithm for a Class of Separable Nonlinear Models

Online Identification of Nonlinear Systems With Separable Structure

Recursive Least Squares Support Vector Regression

Robust Variable Projection Algorithm for the Identification of Separable Nonlinear Models

A Regularized Variable Projection Algorithm for Separable Nonlinear Least-Squares Problems.

Modified Gram–Schmidt Method-Based Variable Projection Algorithm for Separable Nonlinear Models

An Iterative Implementation of Variable Projection for Separable Nonlinear Optimization Problems

Accelerating Variable Projection Algorithm by Aitken Technique.

Multi-innovation-based online variable projection algorithm for a class of nonlinear models

Variable projection algorithms with sparse constraint for separable nonlinear models

Recursive Closed-Loop Subspace Identification Based on Predictor Markov Parameters

Recursive Least-Squares Method with Membership Functions

Modified recursive least squares algorithm with variable parameters and resetting for time-varying system

An efficient variable projection formulation for separable nonlinear least squares problems.

Separable nonlinear least squares: the variable projection method and its applications

Insights Into Algorithms for Separable Nonlinear Least Squares Problems

An Efficient Algorithm for Ill-Conditioned Separable Nonlinear Least Squares

Nuisance Parameter Estimation Algorithms for Separable Nonlinear Models

Embedded Point Iteration Based Recursive Algorithm for Online Identification of Nonlinear Regression Models

Variable Projection Algorithms: Theoretical Insights and A Novel Approach for Problems with Large Residual

A Novel RSMI Based on Regression and Natural Power Method