Abstract:We develop a novel and general framework for reduced-bias $M$-estimation from asymptotically unbiased estimating functions. The framework relies on an empirical approximation of the bias by a function of derivatives of estimating function contributions. Reduced-bias $M$-estimation operates either implicitly, by solving empirically-adjusted estimating equations, or explicitly, by subtracting the estimated bias from the original $M$-estimates, and applies to models that are partially- or fully-specified, with either likelihoods or other surrogate objectives. Automatic differentiation can be used to abstract away the only algebra required to implement reduced-bias $M$-estimation. As a result, the bias reduction methods we introduce have markedly broader applicability with more straightforward implementation and less algebraic or computational effort than other established bias-reduction methods that require resampling or evaluation of expectations of products of log-likelihood derivatives. If $M$-estimation is by maximizing an objective, then there always exists a bias-reducing penalized objective. That penalized objective relates closely to information criteria for model selection, and can be further enhanced with plug-in penalties to deliver reduced-bias $M$-estimates with extra properties, like finiteness in models for categorical data. The reduced-bias $M$-estimators have the same asymptotic distribution as the original $M$-estimators, and, hence, standard procedures for inference and model selection apply unaltered with the improved estimates. We demonstrate and assess the properties of reduced-bias $M$-estimation in well-used, prominent modelling settings of varying complexity.

A New Class of Reduced-Bias Generalized Hill Estimators

A class of unbiased location invariant Hill-type estimators for heavy tailed distributions

Robust Estimation and Shrinkage in Ultrahigh Dimensional Expectile Regression with Heavy Tails and Variance Heterogeneity

Non-parametric Tests for the Tail Equivalence Via Empirical Likelihood

Reduced-bias estimation of the residual dependence index with unnamed marginals

Penalized bias reduction in extreme value estimation for censored Pareto-type data, and long-tailed insurance applications

Inference for High-Dimensional Linear Expectile Regression with De-Biasing Method

Bias-reduced and variance-corrected asymptotic Gaussian inference about extreme expectiles

EVIboost for the Estimation of Extreme Value Index under Heterogeneous Extremes

Improvements in the estimation of the Weibull tail coefficient -- a comparative study

Improvements in the estimation of the Weibull tail coefficient: A comparative study

Inference about the tail of a distribution. Improvement on the Hill estimator

Extreme expectile estimation for short-tailed data

Divergence based robust estimation of the tail index through an exponential regression model

Small-sample likelihood inference in extreme-value regression models

Bias Reduction for High Quantiles

Bias Reduced Peaks over Threshold Tail Estimation

Estimation and Inference of Extremal Quantile Treatment Effects for Heavy-Tailed Distributions

Efficient Estimation in Extreme Value Regression Models of Hedge Fund Tail Risks

Empirical bias-reducing adjustments to estimating functions

Maximum likelihood estimators based on the block maxima method