Abstract:Instead of minimizing the sum of all $n$ squared residuals as the classical least squares (LS) does, Rousseeuw (1984) proposed to minimize the sum of $h$ ($n/2 \leq h < n$) smallest squared residuals, the resulting estimator is called least trimmed squares (LTS). The idea of the LTS is simple but its computation is challenging since no LS-type analytical computation formula exists anymore. Attempts had been made since its presence, the feasible solution algorithm (Hawkins (1994)), fastlts.f (Rousseeuw and Van Driessen (1999)), and FAST-LTS (Rousseeuw and Van Driessen (2006)), among others, are promising approximate algorithms. The latter two have been incorporated into R function ltsReg by Valentin Todorov. These algorithms utilize combinatorial- or subsampling- approaches. With the great software accessibility and fast speed, the LTS, enjoying many desired properties, has become one of the most popular robust regression estimators across multiple disciplines. This article proposes analytic approaches -- employing first-order derivative (gradient) and second-order derivative (Hessian matrix) of the objective function. Our approximate algorithms for the LTS are vetted in synthetic and real data examples. Compared with ltsReg -- the benchmark in robust regression and well-known for its speed, our algorithms are comparable (and sometimes even favorable) with respect to both speed and accuracy criteria. Other major contributions include (i) originating the uniqueness and the strong and Fisher consistency at empirical and population settings respectively; (ii) deriving the influence function in a general setting; (iii) re-establishing the asymptotic normality (consequently root-n consistency) of the estimator with a neat and general approach.

An Efficient Hessian Based Algorithm for Singly Linearly and Box Constrained Least Squares Regression.

A New Algorithm For Unconstrained Optimization Problem With The Form Of Sum Of Squares Minimization

Convergence of Projected Subgradient Method with Sparse or Low-Rank Constraints

Efficient sparse Hessian based algorithms for the clustered lasso problem.

A Dual Semismooth Newton Based Augmented Lagrangian Method for Large-Scale Linearly Constrained Sparse Group Square-Root Lasso Problems

An efficient Hessian based algorithm for solving large-scale sparse group Lasso problems

An Efficient Algorithm for a Class of Large-Scale Support Vector Machines Exploiting Hidden Sparsity.

The Convergence of Least-Squares Progressive Iterative Approximation for Singular Least-Squares Fitting System.

An Efficient Algorithm for Ill-Conditioned Separable Nonlinear Least Squares

A Family of Scaled Factorized Broyden-Like Methods for Nonlinear Least Squares Problems

An improved conjugate gradient scheme to the solution of least squares SVM

An accelerated active-set algorithm for a quadratic semidefinite program with general constraints

SNAP: A semismooth Newton algorithm for pathwise optimization with optimal local convergence rate and oracle properties

Calibrating Least Squares Semidefinite Programming with Equality and Inequality Constraints

Generalized Newton–Raphson Algorithm for High Dimensional LASSO Regression

A sparse semismooth Newton based augmented Lagrangian method for large-scale support vector machines

A Fast Iterative Single Data Approach to Training Unconstrained Least Squares Support Vector Machines

Semi-Smooth Newton Algorithm for Non-Convex Penalized Linear Regression

A Fast Scale-Invariant Algorithm for Non-negative Least Squares with Non-negative Data

New algorithms for computing the least trimmed squares estimator

A structured diagonal Hessian approximation method with evaluation complexity analysis for nonlinear least squares