Minimax Estimation of Parameter in a Family of Distributions under Two Different Loss Functions

任海平,宋允全

DOI: https://doi.org/10.3969/j.issn.1673-8020.2009.03.003

2009-01-01

Abstract:Bayesian and Minimax estimators of parameter in a Family of distributions are given under two different loss functions which are squared log error and entropy loss functions.

What problem does this paper attempt to address?

Bayesian and minimax estimators of loss

Christine Allard,Éric Marchand

2023-12-19

Abstract:We study the problem of loss estimation that involves for an observable $X \sim f_{\theta}$ the choice of a first-stage estimator $\hat{\gamma}$ of $\gamma(\theta)$, incurred loss $L=L(\theta, \hat{\gamma})$, and the choice of a second-stage estimator $\hat{L}$ of $L$. We consider both: (i) a sequential version where the first-stage estimate and loss are fixed and optimization is performed at the second-stage level, and (ii) a simultaneous version with a Rukhin-type loss function designed for the evaluation of $(\hat{\gamma}, \hat{L})$ as an estimator of $(\gamma, L)$. We explore various Bayesian solutions and provide minimax estimators for both situations (i) and (ii). The analysis is carried out for several probability models, including multivariate normal models $N_d(\theta, \sigma^2 I_d)$ with both known and unknown $\sigma^2$, Gamma, univariate and multivariate Poisson, and negative binomial models, and relates to different choices of the first-stage and second-stage losses. The minimax findings are achieved by identifying least favourable of sequence of priors and depend critically on particular Bayesian solution properties, namely situations where the second-stage estimator $\hat{L}(x)$ is constant as a function of $x$.

Methodology,Other Statistics,Statistics Theory
Maximum Score Estimation of Preference Parameters for a Binary Choice Model under Uncertainty

Le-Yu Chen,Sokbae Lee,Myung Jae Sung

DOI: https://doi.org/10.48550/arXiv.1304.5747

2013-12-03

Abstract:This paper develops maximum score estimation of preference parameters in the binary choice model under uncertainty in which the decision rule is affected by conditional expectations. The preference parameters are estimated in two stages: we estimate conditional expectations nonparametrically in the first stage and then the preference parameters in the second stage based on Manski (1975, 1985)'s maximum score estimator using the choice data and first stage estimates. The paper establishes consistency and derives rate of convergence of the two-stage maximum score estimator. Moreover, the paper also provides sufficient conditions under which the two-stage estimator is asymptotically equivalent in distribution to the corresponding single-stage estimator that assumes the first stage input is known. These results are of independent interest for maximum score estimation with nonparametrically generated regressors. The paper also presents some Monte Carlo simulation results for finite-sample behavior of the two-stage estimator.

Methodology,Statistics Theory
Two New Families of Local Asymptotically Minimax Lower Bounds in Parameter Estimation

Neri Merhav

2024-09-19

Abstract:We propose two families of asymptotically local minimax lower bounds on parameter estimation performance. The first family of bounds applies to any convex, symmetric loss function that depends solely on the difference between the estimate and the true underlying parameter value (i.e., the estimation error), whereas the second is more specifically oriented to the moments of the estimation error. The proposed bounds are relatively easy to calculate numerically (in the sense that their optimization is over relatively few auxiliary parameters), yet they turn out to be tighter (sometimes significantly so) than previously reported bounds that are associated with similar calculation efforts, across a variety of application examples. In addition to their relative simplicity, they also have the following advantages: (i) Essentially no regularity conditions are required regarding the parametric family of distributions; (ii) The bounds are local (in a sense to be specified); (iii) The bounds provide the correct order of decay as functions of the number of observations, at least in all examples examined; (iv) At least the first family of bounds extends straightforwardly to vector parameters.

Statistics Theory,Information Theory
Loss modeling with many-parameter distributions

Erik Bølviken,Ingrid Hobæk Haff

DOI: https://doi.org/10.1080/03461238.2024.2309987

IF: 1.7821

2024-02-07

Scandinavian Actuarial Journal

Abstract:It is argued that many-parameter families of loss distributions may work even with limited amounts of historical data. A restriction to unimodality works as a stabilizer, which makes fitted distributions much more stable than their parameters. We propose Box-Cox transformed Gamma and Burr variables. Those are models with three or four parameters with many of the traditional two-parameter families as special cases, and there are well-defined distributions at the boundaries of the parameter space, which is important for stability. The approach is evaluated with model error defined though the theory of misspecification in statistics. It is shown that such error is drastically reduced when a third or fourth parameter is added without increasing the random error more than a little. It is pointed out that the approach may be a suitable starting point for completely automatic procedures.

social sciences, mathematical methods,mathematics, interdisciplinary applications,statistics & probability
The Superiority of the Bayes Linear Unbiased Minimum Variance Estimator under Balanced Loss Function

Miao Bai-qi

2011-01-01

Abstract:The superiority of the Bayes linear unbiased minimum variance(BLUMV) estimator with respect to the optimally weighted least square(OWLS) estimator of unknown parameter was studied in terms of the balanced loss risk function criterion,and the two estimators can converge to the same one under a certain condition.The superiority of the BLUMV estimator over the OWLS estimator was studied under predictive Pitman closeness(PRPC) criterion.
RELIABILITY ANALYSIS FOR THE TWO-PARAMETER PARETO DISTRIBUTION UNDER RECORD VALUES

Liang Wang,Yimin Shi,P.S. Chang

DOI: https://doi.org/10.14317/jami.2011.29.5_6.1435

2011-01-01

Abstract:In this paper the estimation of the parameters as well as sur- vival and hazard functions are presented for the two-parameter Pareto distribution by using Bayesian and non-Bayesian approaches under upper record values. Maximum likelihood estimation (MLE) and interval esti- mation are derived for the parameters. Bayes estimators of reliability per- formances are obtained under symmetric (Squared error) and asymmetric (Linex and general entropy (GE)) losses, when two parameters have dis- crete and continuous priors, respectively. Finally, two numerical examples with real data set and simulated data, are presented to illustrate the pro- posed method. An algorithm is introduced to generate records data, then a simulation study is performed and difierent estimates results are compared.
On continuous distribution functions, minimax and best invariant estimators, and integrated balanced loss functions

Mohammad Jafari Jozani,Alexandre Leblanc,Eric Marchand

DOI: https://doi.org/10.48550/arXiv.1308.1815

2013-08-08

Abstract:We consider the problem of estimating a continuous distribution function $F$, as well as meaningful functions $\tau(F)$ under a large class of loss functions. We obtain best invariant estimators and establish their minimaxity for Hölder continuous $\tau$'s and strict bowl-shaped losses with a bounded derivative. We also introduce and motivate the use of integrated balanced loss functions which combine the criteria of an integrated distance between a decision $d$ and $F$, with the proximity of $d$ with a target estimator $d_0$. Moreover, we show how the risk analysis of procedures under such an integrated balanced loss relates to a dual risk analysis under an "unbalanced" loss, and we derive best invariant estimators, minimax estimators, risk comparisons, dominance and inadmissibility results. Finally, we expand on various illustrations and applications relative to maxima-nomination sampling, median-nomination sampling, and a case study related to bilirubin levels in the blood of babies suffering from jaundice.

Statistics Theory
Improved parameter estimation for a family of exponential distributions

S.B. Kologrivova,E.A. Pchelintsev

2023-08-05

Abstract:In this paper, we consider the problem of parameter estimating for a family of exponential distributions. We develop the improved estimation method, which generalized the James--Stein approach for a wide class of distributions. The proposed estimator dominates the classical maximum likelihood estimator under the quadratic risk. The estimating procedure is applied to special cases of distributions. The numerical simulations results are given.

Statistics Theory
Second-Order Characterization of Minimax Parameter Estimation in Restricted Parameter Space

Tianren Peng,Xinyi Tong,Shao-Lun Huang

DOI: https://doi.org/10.1109/isit57864.2024.10619636

2024-01-01

Abstract:Estimating unknown parameters in restricted parameter space is an important problem with applications in communication, statistics, and machine learning. In this paper, we adopt the conventional minimax formulation to investigate such problems. In particular, we focus on studying the second-order characterizations of the minimax risk in the asymptotic regime. We first show that the second-order convergence rate of the minimax risk depends on the local flatness of the Fisher information around its global optimum. Then, we demonstrate that the second-order terms can be computed by solving certain ordinary differential equations, where the coefficients of the second-order terms can be explicitly expressed in some cases. Finally, the estimators achieving the minimax risk are also given, which provides potential guidance for machine learning designs.
Minimax rates of l p-losses for high-dimensional linear regression models with additive measurement errors over l q-balls

Xin Li,Dongya Wu

2019-01-01

Abstract:We study minimax rates for high-dimensional linear regression with additive errors under the lp (1 ≤ p < ∞)-losses, where the regression parameter is of weak sparsity. Our lower and upper bounds agree up to constant factors, implying that the proposed estimator is minimax optimal.
Two-stage and purely sequential minimum risk point estimation of the scale parameter of a family of distributions under modified LINEX loss plus sampling cost

Neeraj Joshi,Sudeep R. Bapat,Raghu Nandan Sengupta

DOI: https://doi.org/10.1007/s00184-024-00973-1

IF: 0.96

2024-09-14

Metrika

Abstract:In this research, we present two-stage and purely sequential methodologies for estimating the scale parameter of the Moore and Bilikam family of lifetime distributions (see Moore and Bilikam in IEEE Trans Reliabil 27:64–67, 1978). We propose our methodologies under the minimum risk point estimation setup, whereby we consider the modified LINEX loss function plus non linear sampling cost. We study some interesting exact distributional properties associated with our stopping rules. We also present simulation analyses using Weibull distribution (special case of the Moore and Bilikam family) to check the performance of our two-stage and purely sequential procedures. Finally, we provide a real data set from COVID-19 and analyze it using the Weibull model in support of the practical utility of our proposed two-stage methodology.

statistics & probability
Robust explicit estimation of the log-logistic distribution with applications

Zhuanzhuan Ma,Min Wang,Chanseok Park

DOI: https://doi.org/10.48550/arXiv.2209.07261

2022-09-15

Abstract:The parameters of the log-logistic distribution are generally estimated based on classical methods such as maximum likelihood estimation, whereas these methods usually result in severe biased estimates when the data contain outliers. In this paper, we consider several alternative estimators, which not only have closed-form expressions, but also are quite robust to a certain level of data contamination. We investigate the robustness property of each estimator in terms of the breakdown point. The finite sample performance and effectiveness of these estimators are evaluated through Monte Carlo simulations and a real-data application. Numerical results demonstrate that the proposed estimators perform favorably in a manner that they are comparable with the maximum likelihood estimator for the data without contamination and that they provide superior performance in the presence of data contamination.

Methodology
On Parameter Estimation in Deviated Gaussian Mixture of Experts

Huy Nguyen,Khai Nguyen,Nhat Ho

2024-06-24

Abstract:We consider the parameter estimation problem in the deviated Gaussian mixture of experts in which the data are generated from $(1 - \lambda^{\ast}) g_0(Y| X)+ \lambda^{\ast} \sum_{i = 1}^{k_{\ast}} p_{i}^{\ast} f(Y|(a_{i}^{\ast})^{\top}X+b_i^{\ast},\sigma_{i}^{\ast})$, where $X, Y$ are respectively a covariate vector and a response variable, $g_{0}(Y|X)$ is a known function, $\lambda^{\ast} \in [0, 1]$ is true but unknown mixing proportion, and $(p_{i}^{\ast}, a_{i}^{\ast}, b_{i}^{\ast}, \sigma_{i}^{\ast})$ for $1 \leq i \leq k^{\ast}$ are unknown parameters of the Gaussian mixture of experts. This problem arises from the goodness-of-fit test when we would like to test whether the data are generated from $g_{0}(Y|X)$ (null hypothesis) or they are generated from the whole mixture (alternative hypothesis). Based on the algebraic structure of the expert functions and the distinguishability between $g_0$ and the mixture part, we construct novel Voronoi-based loss functions to capture the convergence rates of maximum likelihood estimation (MLE) for our models. We further demonstrate that our proposed loss functions characterize the local convergence rates of parameter estimation more accurately than the generalized Wasserstein, a loss function being commonly used for estimating parameters in the Gaussian mixture of experts.

Machine Learning
Parameter Estimation of Birnbaum-Saunders Distribution under Competing Risks Using the Quantile Variant of the Expectation-Maximization Algorithm

Chanseok Park,Min Wang

DOI: https://doi.org/10.3390/math12111757

IF: 2.4

2024-06-06

Mathematics

Abstract:Competing risks models, also known as weakest-link models, are utilized to analyze diverse strength distributions exhibiting multi-modality, often attributed to various types of defects within the material. The weakest-link theory posits that a material's fracture is dictated by its most severe defect. However, multimodal problems can become intricate due to potential censoring, a common constraint stemming from time and cost limitations during experiments. Additionally, determining the mode of failure can be challenging due to factors like the absence of suitable diagnostic tools, costly autopsy procedures, and other obstacles, collectively referred to as the masking problem. In this paper, we investigate the distribution of strength for multimodal failures with censored data. We consider both full and partial maskings and present an EM-type parameter estimate for the Birnbaum-Saunders distribution under competing risks. We compare the results with those obtained from other distributions, such as lognormal, Weibull, and Wald (inverse-Gaussian) distributions. The effectiveness of the proposed method is demonstrated through two illustrative examples, as well as an analysis of the sensitivity of parameter estimates to variations in starting values.

mathematics
Minimax rates of convergence for nonparametric location-scale models

Bingxin Zhao,Yuhong Yang

2023-07-04

Abstract:This paper studies minimax rates of convergence for nonparametric location-scale models, which include mean, quantile and expectile regression settings. Under Hellinger differentiability on the error distribution and other mild conditions, we show that the minimax rate of convergence for estimating the regression function under the squared $L_2$ loss is determined by the metric entropy of the nonparametric function class. Different error distributions, including asymmetric Laplace distribution, asymmetric connected double truncated gamma distribution, connected normal-Laplace distribution, Cauchy distribution and asymmetric normal distribution are studied as examples. Applications on low order interaction models and multiple index models are also given.

Statistics Theory
Robust estimation of a regression function in exponential families

Yannick Baraud,Juntong Chen

DOI: https://doi.org/10.1016/j.jspi.2024.106167

IF: 1.095

2024-03-26

Journal of Statistical Planning and Inference

Abstract:We observe n pairs of independent (but not necessarily i.i.d.) random variables X1=(W1,Y1),...,Xn=(Wn,Yn) and tackle the problem of estimating the conditional distributions Qi⋆(wi) of Yi given Wi=wi for all i∈{1,...,n} . Even though these might not be true, we base our estimator on the assumptions that the data are i.i.d. and the conditional distributions of Yi given Wi=wi belong to a one parameter exponential family Q ̄ with parameter space given by an interval I . More precisely, we pretend that these conditional distributions take the form Qθ(wi)∈Q ̄ for some θ that belongs to a VC-class Θ ̄ of functions with values in I . For each i∈{1,...,n} , we estimate Qi⋆(wi) by a distribution of the same form, i.e. Qθ̂(wi)∈Q ̄ , where θ̂=θ̂(X1,...,Xn) is a well-chosen estimator with values in Θ ̄ . We establish non-asymptotic exponential inequalities for the upper deviations of a Hellinger-type distance between the true conditional distributions of the data and the estimated one based on the exponential family Q ̄ and the class of functions Θ ̄ we chose. We show that our estimation strategy is robust to model misspecification, contamination and the presence of outliers. Besides, when the data are truly i.i.d., the exponential family Q ̄ is suitably parametrized and the conditional distributions Qi⋆(wi) of the form Qθ⋆(wi)∈Q ̄ for some unknown Hölderian function θ⋆ with values in I , we prove that the estimator θ̂ of θ⋆ is minimax (up to a logarithmic factor). Finally, we provide an algorithm for calculating θ̂ when Θ ̄ is a VC-class of functions of low or moderate dimension and we carry out a simulation study to compare its performance to that of the MLE and median-based estimators. The proof of our main result relies on an upper bound, with explicit numerical constants, on the expectation of the supremum of an empirical process over a VC-subgraph class. This bound can be of independent interest.

statistics & probability
High-probability minimax lower bounds

Tianyi Ma,Kabir A. Verchand,Richard J. Samworth

2024-07-04

Abstract:The minimax risk is often considered as a gold standard against which we can compare specific statistical procedures. Nevertheless, as has been observed recently in robust and heavy-tailed estimation problems, the inherent reduction of the (random) loss to its expectation may entail a significant loss of information regarding its tail behaviour. In an attempt to avoid such a loss, we introduce the notion of a minimax quantile, and seek to articulate its dependence on the quantile level. To this end, we develop high-probability variants of the classical Le Cam and Fano methods, as well as a technique to convert local minimax risk lower bounds to lower bounds on minimax quantiles. To illustrate the power of our framework, we deploy our techniques on several examples, recovering recent results in robust mean estimation and stochastic convex optimisation, as well as obtaining several new results in covariance matrix estimation, sparse linear regression, nonparametric density estimation and isotonic regression. Our overall goal is to argue that minimax quantiles can provide a finer-grained understanding of the difficulty of statistical problems, and that, in wide generality, lower bounds on these quantities can be obtained via user-friendly tools.

Statistics Theory,Information Theory,Machine Learning
Parameter estimation of inverse Weibull distribution under competing risks based on the expectation–maximization algorithm

Refah Alotaibi,Hoda Rezk,Chanseok Park

DOI: https://doi.org/10.1002/qre.3599

2024-06-11

Quality and Reliability Engineering International

Abstract:A system consisting of interconnected components in series is under consideration. This research focuses on estimating the parameters of this system for incomplete lifetime data within the framework of competing risks, employing an underlying inverse Weibull distribution. While one popular method for parameter estimation involves the Newton–Raphson (NR) technique, its sensitivity to initial value selection poses a significant drawback, often resulting in convergence failures. Therefore, this paper opts for the expectation–maximization (EM) algorithm. In competing risks scenarios, the precise cause of failure is frequently unidentified, and these issues can be further complicated by potential censoring. Thus, incompleteness may arise due to both censoring and masking. In this study, we present the EM‐type parameter estimation and demonstrate its superiority over parameter estimation based on the NR method. Two illustrative examples are provided. The proposed method is compared with the existing Weibull competing risks model, revealing the superiority of our approach. Through Monte Carlo simulations, we also examine the sensitivity of the initial value selection for both the NR‐type method and our proposed method.

engineering, industrial, multidisciplinary,operations research & management science
A New Class of Estimators Based on a General Relative Loss Function

Tao Hu,Baosheng Liang

DOI: https://doi.org/10.3390/math9101138

IF: 2.4

2021-01-01

Mathematics

Abstract:Motivated by the relative loss estimator of the median, we propose a new class of estimators for linear quantile models using a general relative loss function defined by the Box–Cox transformation function. The proposed method is very flexible. It includes a traditional quantile regression and median regression under the relative loss as special cases. Compared to the traditional linear quantile estimator, the proposed estimator has smaller variance and hence is more efficient in making statistical inferences. We show that, in theory, the proposed estimator is consistent and asymptotically normal under appropriate conditions. Extensive simulation studies were conducted, demonstrating good performance of the proposed method. An application of the proposed method in a prostate cancer study is provided.
Minimax Linear Regression under the Quantile Risk

Ayoub El Hanchi,Chris J. Maddison,Murat A. Erdogdu

2024-06-18

Abstract:We study the problem of designing minimax procedures in linear regression under the quantile risk. We start by considering the realizable setting with independent Gaussian noise, where for any given noise level and distribution of inputs, we obtain the exact minimax quantile risk for a rich family of error functions and establish the minimaxity of OLS. This improves on the known lower bounds for the special case of square error, and provides us with a lower bound on the minimax quantile risk over larger sets of distributions. Under the square error and a fourth moment assumption on the distribution of inputs, we show that this lower bound is tight over a larger class of problems. Specifically, we prove a matching upper bound on the worst-case quantile risk of a variant of the recently proposed min-max regression procedure, thereby establishing its minimaxity, up to absolute constants. We illustrate the usefulness of our approach by extending this result to all $p$-th power error functions for $p \in (2, \infty)$. Along the way, we develop a generic analogue to the classical Bayesian method for lower bounding the minimax risk when working with the quantile risk, as well as a tight characterization of the quantiles of the smallest eigenvalue of the sample covariance matrix.

Statistics Theory,Machine Learning

Minimax Estimation of Parameter in a Family of Distributions under Two Different Loss Functions

Bayesian and minimax estimators of loss

Maximum Score Estimation of Preference Parameters for a Binary Choice Model under Uncertainty

Two New Families of Local Asymptotically Minimax Lower Bounds in Parameter Estimation

Loss modeling with many-parameter distributions

The Superiority of the Bayes Linear Unbiased Minimum Variance Estimator under Balanced Loss Function

RELIABILITY ANALYSIS FOR THE TWO-PARAMETER PARETO DISTRIBUTION UNDER RECORD VALUES

On continuous distribution functions, minimax and best invariant estimators, and integrated balanced loss functions

Improved parameter estimation for a family of exponential distributions

Second-Order Characterization of Minimax Parameter Estimation in Restricted Parameter Space

Minimax rates of l p-losses for high-dimensional linear regression models with additive measurement errors over l q-balls

Two-stage and purely sequential minimum risk point estimation of the scale parameter of a family of distributions under modified LINEX loss plus sampling cost

Robust explicit estimation of the log-logistic distribution with applications

On Parameter Estimation in Deviated Gaussian Mixture of Experts

Parameter Estimation of Birnbaum-Saunders Distribution under Competing Risks Using the Quantile Variant of the Expectation-Maximization Algorithm

Minimax rates of convergence for nonparametric location-scale models

Robust estimation of a regression function in exponential families

High-probability minimax lower bounds

Parameter estimation of inverse Weibull distribution under competing risks based on the expectation–maximization algorithm

A New Class of Estimators Based on a General Relative Loss Function

Minimax Linear Regression under the Quantile Risk