Abstract:Standard regression techniques model only the mean of the response variable. Quantile regression (QR) is more powerful in that it depicts a comprehensive relationship between the response variable and independent covariates at different quantiles. It is particularly useful for non-normally distributed data with skewness or heterogeneity, which appear routinely in many scientific fields, such as economics, finance, public health and biology. Although its theory has been well developed in the literature, its computation in big data still faces multiple challenges, especially for vertically stored big data in modern distributed environments, where communication efficiency and security are usually the primary considerations. While the popular alternating direction method of multipliers (ADMM) provides a general computational solution, its slow convergence becomes a bottleneck when communication cost dominates local computational consumption, such as Internet of Things (IoT) networks. Motivated by the residual projection technique, in this paper we propose an innovative iterative parallel framework, PIQR, that converges faster and has a more secure data transmission plan, and establish its convergence property. This framework is further extended to composite quantile regression (CQR), a modified QR technique that improves estimation efficiency at extreme quantiles. Simulation studies show that both the ADMM-based method and the PIQR enjoy favorable estimation accuracy in distributed environments. While PIQR is inferior to the ADMM-based method at local computation, it requires much fewer iterations to achieve convergence, and hence significantly improves the overall computational efficiency when communication cost is the dominating factor. Moreover, PIQR transmits only data involving the residual information between different machines, and can better prevent the leakage of important data information compared with the ADMM-based method.

Approximate nonparametric quantile regression in reproducing kernel Hilbert spaces via random projection

Communication-Efficient Nonparametric Quantile Regression via Random Features

Optimal Prediction of Quantile Functional Linear Regression in Reproducing Kernel Hilbert Spaces

Fast quantile regression in reproducing kernel Hilbert space

Partially linear functional quantile regression in a reproducing kernel Hilbert space

Sparse high-dimensional semi-nonparametric quantile regression in a reproducing kernel Hilbert space

Optimal Kernel Quantile Learning with Random Features

A Variant of K Nearest Neighbor Quantile Regression

fastkqr: A Fast Algorithm for Kernel Quantile Regression

The Regression Estimation with Random Projection

Quantile Regression With L(1)-Regularization And Gaussian Kernels

Functional Quantile Regression with Missing Data in Reproducing Kernel Hilbert Space

Almost-sure quasi-optimal approximation in reproducing kernel Hilbert spaces

Multi-Layer Kernel Machines: Fast and Optimal Nonparametric Regression with Uncertainty Quantification

Universality of kernel random matrices and kernel regression in the quadratic regime

Randomized Quantile Regression Estimation for Heteroskedastic Non Parametric Model

Sketched quantile additive functional regression

Approximation and generalization properties of the random projection classification method

On the Probabilistic Approximation in Reproducing Kernel Hilbert Spaces

Structured kernel quantile regression

Residual projection for quantile regression in vertically partitioned big data