Abstract:This paper addresses the challenge of integrating sequentially arriving data within the quantile regression framework, where the number of features is allowed to grow with the number of observations, the horizon is unknown, and memory is limited. We employ stochastic sub-gradient descent to minimize the empirical check loss and study its statistical properties and regret performance. In our analysis, we unveil the delicate interplay between updating iterates based on individual observations versus batches of observations, revealing distinct regularity properties in each scenario. Our method ensures long-term optimal estimation irrespective of the chosen update strategy. Importantly, our contributions go beyond prior works by achieving exponential-type concentration inequalities and attaining optimal regret and error rates that exhibit only \textsf{ short-term} sensitivity to initial errors. A key insight from our study is the delicate statistical analyses and the revelation that appropriate stepsize schemes significantly mitigate the impact of initial errors on subsequent errors and regrets. This underscores the robustness of stochastic sub-gradient descent in handling initial uncertainties, emphasizing its efficacy in scenarios where the sequential arrival of data introduces uncertainties regarding both the horizon and the total number of observations. Additionally, when the initial error rate is well-controlled, there is a trade-off between short-term error rate and long-term optimality. Due to the lack of delicate statistical analysis for squared loss, we also briefly discuss its properties and proper schemes. Extensive simulations support our theoretical findings.

Sparse quantile regression

Sparse Estimation Via ℓ_q Optimization Method in High-Dimensional Linear Regression

Sparse Estimation Via Lower-Order Penalty Optimization Methods in High-Dimensional Linear Regression.

A sparse PAC-Bayesian approach for high-dimensional quantile prediction

Sparse and Low-Rank Matrix Quantile Estimation with Application to Quadratic Regression

A Sparse Approach for High-Dimensional Data with Heavy-Tailed Noise

Nearly Semiparametric Efficient Estimation of Quantile Regression

A Smoothing Iterative Method for Quantile Regression with Nonconvex \begin{document}$ \ell_p $\end{document} Penalty

Generalized ℓ1-Penalized Quantile Regression with Linear Constraints.

Optimal Subsampling for $$L_p$$-Quantile Regression Via Decorrelated Score

Nonparametric Quantile Regression: Non-Crossing Constraints and Conformal Prediction

Minimax Linear Regression under the Quantile Risk

SCAD‐penalized Quantile Regression for High‐dimensional Data Analysis and Variable Selection

Incorporating Graphical Structure of Predictors in Sparse Quantile Regression

Regularization Strategies for Quantile Regression

Sparse and Debiased Lasso Estimation and Inference for High-Dimensional Composite Quantile Regression with Distributed Data

Convergence Rate for Nonparametric Quantile Regression with a Total Variation Penalty

Online Quantile Regression

Communication‐efficient Low‐dimensional Parameter Estimation and Inference for High‐dimensional Lp$$ {l}^p $$‐quantile Regression

Communication-efficient Estimation of High-Dimensional Quantile Regression

An Empirical Quantile Estimation Approach for Chance-Constrained Nonlinear Optimization Problems