Abstract:In recent years, distributed statistical models have received increasing attention for large-scale data analysis. On the one hand, data sets come from multiple data sources, and are stored in different locations due to limited bandwidth and storage, or privacy protocols, directly centralizing all data together is impossible. On the other hand, the size of data is so large that it is difficult or inefficient to analyze data together. There are two main research aspects to using distributed statistical models to analyze large-scale data. The first one is to study the statistical convergence rate under some mild assumptions. The second one is to establish fast and efficient optimization algorithms considering the property of the loss function. There is a lot of research on the first aspect, but relatively little research on the second one. Motivated by this, we consider the construction of unified algorithms for distributed linear regression with different losses and regularizers. As a result, we designed two type methods, proximal alternating direction method of multipliers (pADMM) and distributed accelerated proximal gradient method with line-search (DAPGL). In order to demonstrate the efficiency of the proposed algorithms, we perform numerical experiments on the distributed Huber-Lasso model and Huber-Group-Lasso model. In view of the numerical results, we can observe that these two algorithms are more competitive than some of state-of-art algorithms. In particular, DAPGL algorithm performs better than pADMM in most cases.

Distributed Learning and Distribution Regression of Coefficient Regularization.

Distributed Semi-supervised Regression Learning with Coefficient Regularization

Coefficient-based regularized distribution regression

Learning Theory of Distributed Regression with Bias Corrected Regularization Kernel Network

Distributed Learning with Indefinite Kernels

Distributed Learning with Multi-Penalty Regularization

Distributed Linear Regression with Compositional Covariates

Distributed Learning with Convex SUM-of -Non-convex Objective

Estimates on Learning Rates for Multi-Penalty Distribution Regression

Optimal Learning Rates for Distribution Regression

Unified algorithms for distributed regularized linear regression model

Regularized Adaptive Huber Matrix Regression and Distributed Learning

Robust Regression for Non-Randomly Distributed Big Data with Application to Nonconvex Penalized Learning

The distributed L1/2 regularization

A Distributed Learning Method for ℓ 1 -Regularized Kernel Machine over Wireless Sensor Networks

Distributed Online Linear Regressions

Distributed Extreme Learning Machine for Nonlinear Learning over Network

A distributed block coordinate descent method for training $l_1$ regularized linear classifiers

Convergence Analysis of Distributed Multi-Penalty Regularized Pairwise Learning

Distributed Semi-Supervised Learning with Kernel Ridge Regression

Distributed Kernel Ridge Regression with Communications.