Abstract:This paper addresses the challenge of integrating sequentially arriving data within the quantile regression framework, where the number of features is allowed to grow with the number of observations, the horizon is unknown, and memory is limited. We employ stochastic sub-gradient descent to minimize the empirical check loss and study its statistical properties and regret performance. In our analysis, we unveil the delicate interplay between updating iterates based on individual observations versus batches of observations, revealing distinct regularity properties in each scenario. Our method ensures long-term optimal estimation irrespective of the chosen update strategy. Importantly, our contributions go beyond prior works by achieving exponential-type concentration inequalities and attaining optimal regret and error rates that exhibit only \textsf{ short-term} sensitivity to initial errors. A key insight from our study is the delicate statistical analyses and the revelation that appropriate stepsize schemes significantly mitigate the impact of initial errors on subsequent errors and regrets. This underscores the robustness of stochastic sub-gradient descent in handling initial uncertainties, emphasizing its efficacy in scenarios where the sequential arrival of data introduces uncertainties regarding both the horizon and the total number of observations. Additionally, when the initial error rate is well-controlled, there is a trade-off between short-term error rate and long-term optimality. Due to the lack of delicate statistical analysis for squared loss, we also briefly discuss its properties and proper schemes. Extensive simulations support our theoretical findings.

Distributed Statistical Estimation in Quantile Regression over a Network

Distributed Bootstrap Simultaneous Inference for High-Dimensional Quantile Regression

Distributed Quantile Regression over Sensor Networks

Distributed Online Quantile Regression over Networks with Quantized Communication.

Distributed High-dimensional Regression under a Quantile Loss Function

Distributed High-Dimensional Quantile Regression: Estimation Efficiency and Support Recovery

Online Quantile Regression

Distributed Optimization with Quantized Gradient Descent

Communication-efficient estimation and inference for high-dimensional quantile regression based on smoothed decorrelated score

Distributed quantile regression for massive heterogeneous data

A Note on Distributed Quantile Regression by Pilot Sampling and One-Step Updating

Quantized Distributed Gradient Tracking Algorithm with Linear Convergence in Directed Networks

Communication-Efficient Nonparametric Quantile Regression via Random Features

Distributed Censored Regression over Networks

Convergence in High Probability of Distributed Stochastic Gradient Descent Algorithms

Fast and Robust Low-Rank Learning over Networks: A Decentralized Matrix Quantile Regression Approach

Convergence Theory of Generalized Distributed Subgradient Method with Random Quantization

Distributed Constrained Optimization With Delayed Subgradient Information Over Time-Varying Network Under Adaptive Quantization

Dynamic Network Quantile Regression Model

Fast networked data selection via distributed smoothed quantile estimation

Neural Networks for Partially Linear Quantile Regression