Abstract:Quantile regression is a fundamental problem in statistical learning motivated by a need to quantify uncertainty in predictions, or to model a diverse population without being overly reductive. For instance, epidemiological forecasts, cost estimates, and revenue predictions all benefit from being able to quantify the range of possible values accurately. As such, many models have been developed for this problem over many years of research in statistics, machine learning, and related fields. Rather than proposing yet another (new) algorithm for quantile regression we adopt a meta viewpoint: we investigate methods for aggregating any number of conditional quantile models, in order to improve accuracy and robustness. We consider weighted ensembles where weights may vary over not only individual models, but also over quantile levels, and feature values. All of the models we consider in this paper can be fit using modern deep learning toolkits, and hence are widely accessible (from an implementation point of view) and scalable. To improve the accuracy of the predicted quantiles (or equivalently, prediction intervals), we develop tools for ensuring that quantiles remain monotonically ordered, and apply conformal calibration methods. These can be used without any modification of the original library of base models. We also review some basic theory surrounding quantile aggregation and related scoring rules, and contribute a few new results to this literature (for example, the fact that post sorting or post isotonic regression can only improve the weighted interval score). Finally, we provide an extensive suite of empirical comparisons across 34 data sets from two different benchmark repositories.

Discussion: A comparison of GAMLSS with quantile regression

Comparison between Highly Complex Location Models and GAMLSS

Flexible Bayesian quantile regression for nonlinear mixed effects models based on the generalized asymmetric Laplace distribution

Global quantile regression

Empirical Likelihood Based Tests for Detecting the Presence of Significant Predictors in Marginal Quantile Regression

Functional data analysis of generalized regression quantiles

MM Algorithms for Statistical Estimation in Quantile Regression

Flexible Model Aggregation for Quantile Regression

Generalized Logistic Models and its orthant tail dependence

A novel grey prediction model based on quantile regression

A New Family of Error Distributions for Bayesian Quantile Regression

Regression via Arbitrary Quantile Modeling

Empirical likelihood ratio test on quantiles under a density ratio model

A comparison of some conformal quantile regression methods

Quantile ratio regression

Quantile Least Squares: A Flexible Approach for Robust Estimation and Validation of Location-Scale Families

Estimating Confidence Intervals and Regions for Quantiles by Monte Carlo Simulation

Looking beyond the mean: quantile regression for comparative physiologists

Nearly Semiparametric Efficient Estimation of Quantile Regression

Extremal Quantile Regression: An Overview

Reprint: Hypothesis testing on high dimensional quantile regression