Abstract:In recent years, distributed statistical models have received increasing attention for large-scale data analysis. On the one hand, data sets come from multiple data sources, and are stored in different locations due to limited bandwidth and storage, or privacy protocols, directly centralizing all data together is impossible. On the other hand, the size of data is so large that it is difficult or inefficient to analyze data together. There are two main research aspects to using distributed statistical models to analyze large-scale data. The first one is to study the statistical convergence rate under some mild assumptions. The second one is to establish fast and efficient optimization algorithms considering the property of the loss function. There is a lot of research on the first aspect, but relatively little research on the second one. Motivated by this, we consider the construction of unified algorithms for distributed linear regression with different losses and regularizers. As a result, we designed two type methods, proximal alternating direction method of multipliers (pADMM) and distributed accelerated proximal gradient method with line-search (DAPGL). In order to demonstrate the efficiency of the proposed algorithms, we perform numerical experiments on the distributed Huber-Lasso model and Huber-Group-Lasso model. In view of the numerical results, we can observe that these two algorithms are more competitive than some of state-of-art algorithms. In particular, DAPGL algorithm performs better than pADMM in most cases.

Fast Algorithms for LS and LAD-Collaborative Regression

The linearized alternating direction method of multipliers for low-rank and fused LASSO matrix regression model

A new penalized least absolute deviation model for high dimensional sparse linear regression and an efficient sequential linear programming algorithm

Estimate-Piloted Regularization and Fast ALS Algorithm for Collaborative Filtering

Least-Square Approximation for a Distributed System.

Robust Regression Shrinkage and Consistent Variable Selection Through the LAD-Lasso

LAD-LASSO: SIMULATION STUDY OF ROBUST REGRESSION IN HIGH DIMENSIONAL DATA

Least Squares Approximation for a Distributed System

A unified consensus-based parallel ADMM algorithm for high-dimensional regression with combined regularizations

Linearized Alternating Direction Method with Adaptive Penalty for Low-Rank Representation

Linearized Alternating Direction Method with Adaptive Penalty for Low-Rank Representation.

A constructive approach to L

Collaboratively Learning Linear Models with Structured Missing Data

Application of LADMM and As-LADMM for a High-Dimensional Partially Linear Model

Unified algorithms for distributed regularized linear regression model

Least squares online linear discriminant analysis

An adaptive shortest-solution guided decimation approach to sparse high-dimensional linear regression

A Fast Solution to the Lasso Problem with Equality Constraints

Efficient sparse Hessian based algorithms for the clustered lasso problem.

AN EXTENDED LINEARIZED ALTERNATING DIRECTION METHOD OF MULTIPLIERS FOR FUSED-LASSO PENALIZED LINEAR REGRESSION

Fast Approximate L_infty Minimization: Speeding Up Robust Regression