Extended Glivenko–Cantelli Theorem in Nonparametric Regression

Fuxia Cheng,Jigao Yan,Lijian Yang

DOI: https://doi.org/10.1080/03610926.2012.700377

2014-01-01

Abstract:In this paper, we consider the uniform strong consistency of the cumulative distribution function estimator in nonparametric regression. We obtain the extended Glivenko–Cantelli theorem for the residual-based empirical distribution function.

What problem does this paper attempt to address?

Asymptotic expansion for nonparametric M-estimator in a nonlinear regression model with long-memory errors

Jia Chen,Degui Li,Zhengyan Lin

DOI: https://doi.org/10.1016/j.jspi.2011.03.025

IF: 1.095

2011-01-01

Journal of Statistical Planning and Inference

Abstract:We consider asymptotic expansion of the nonparametric M-estimator in a fixed-design nonlinear regression model when the errors are generated by long-memory linear processes. Under mild conditions, we show that the nonparametric M-estimator is first-order equivalent to the Nadaraya–Watson (NW) estimator, which implies that the nonparametric M-estimator has the same asymptotic distribution as that of the NW estimator. Furthermore, we study the second-order asymptotic expansion of the nonparametric M-estimator and show that the difference between the nonparametric M-estimator and the NW estimator has a limiting distribution after suitable standardization. The nature of the limiting distribution depends on the range of long-memory parameter α. We also compare the finite sample behavior of the two estimators through a numerical example when the errors are long-memory.
Empirical Likelihood Inference for Nonparametric Regression Functions with Functional Stationary Ergodic Data

Xian-Zhu Xiong,Zheng-Yan Lin

DOI: https://doi.org/10.1080/03610926.2011.630766

2013-01-01

Abstract:In this article, we employ the empirical likelihood method to construct confidence intervals for nonparametric regression functions with functional stationary ergodic data. Under some wild conditions, it is proved that the empirical likelihood ratio statistic is asymptotically chi-squared distributed.
Non-parametric Tests for the Tail Equivalence Via Empirical Likelihood

Yuexiang Jiang,Haoze Sun,Yi Zhang,Huaigang Long

DOI: https://doi.org/10.1080/03610926.2016.1242736

2017-01-01

Abstract:In this paper, the problem of whether the left tail and the right tail of a distribution share the same extreme value index (EVI) is addressed and we propose two different test statistics. The first one is based on the result of the joint asymptotic normality of the two Hill estimators for the EVIs of both tails. And therefore, we can construct a quotient-type test statistic, which is asymptotic (2)(1) distributed after some standardization. The second test statistic proposed in this paper is inspired by the two-sample empirical likelihood methodology, and we prove its non parametric version of Wilk's theorem. At last, we compare the efficiencies of our two test statistics and the maximum likelihood (ML) ratio test statistic proposed by Jondeau and Rockinger (2003) in terms of empirical first type error and power through a number of simulation studies, which indicate that the performance of the ML ratio test statistic is worse than our two test statistics in most cases.
Consistent Specification Testing Via Nonparametric Series Regression

YM HONG,H WHITE

DOI: https://doi.org/10.2307/2171724

IF: 6.1

1995-01-01

Econometrica

Abstract:This paper proposes two consistent one-sided specification tests for parametric regression models, one based on the sample covariance between the residual from the parametric model and the discrepancy between the parametric and nonparametric fitted values; the other based on the difference in sums of squared residuals between the parametric and nonparametric models. We estimate the nonparametric model by series regression. The new test statistics converge in distribution to a unit normal under correct specification and grow to infinity faster than the parametric rate (n(-1/2)) under misspecification, while avoiding weighting, sample splitting, and non-nested testing procedures used elsewhere in the literature. Asymptotically, our tests can be viewed as a test of the joint hypothesis that the true parameters of a series regression model are zero, where the dependent variable is the residual from the parametric model, and the series terms are functions of the explanatory variables, chosen so as to support nonparametric estimation of a conditional expectation. We specifically consider Fourier series and regression splines, and present a Monte Carlo study of the finite sample performance of the new tests in comparison to consistent tests of Bierens (1990), Eubank and Spiegelman (1990), Jayasuriya (1990), Wooldridge (1992), and Yatchew (1992); the results show the new tests have good power, performing quite well in some situations. We suggest a joint Bonferroni procedure that combines a new test with those of Bierens and Wooldridge to capture the best features of the three approaches.
Strong Consistency of the Internal Estimator of Nonparametric Regression with Dependent Data

Jia Shen,Yuan Xie

DOI: https://doi.org/10.1016/j.spl.2013.04.027

IF: 0.718

2013-01-01

Statistics & Probability Letters

Abstract:In this paper, the strong consistency of the multivariate internal nonparametric estimator is investigated under strong mixing dependence assumption. This estimator is particularly easy to use when we model the regression function by additive nonparametric structure. The pointwise strong consistency and its rate are given as well as that over a compact set, under suitable conditions.
Strong Limit Theorems for Extended Independent Random Variables and Extended Negatively Dependent Random Variables under Sub-Linear Expectations

Zhang Li-Xin

DOI: https://doi.org/10.1007/s10473-022-0203-z

2022-01-01

Acta Mathematica Scientia

Abstract:Limit theorems for non-additive probabilities or non-linear expectations are challenging issues which have attracted a lot of interest recently. The purpose of this paper is to study the strong law of large numbers and the law of the iterated logarithm for a sequence of random variables in a sub-linear expectation space under a concept of extended independence which is much weaker and easier to verify than the independence proposed by Peng [20]. We introduce a concept of extended negative dependence which is an extension of the kind of weak independence and the extended negative independence relative to classical probability that has appeared in the recent literature. Powerful tools such as moment inequality and Kolmogorov's exponential inequality are established for these kinds of extended negatively independent random variables, and these tools improve a lot upon those of Chen, Chen and Ng [1]. The strong law of large numbers and the law of iterated logarithm are also obtained by applying these inequalities.
Asymptotic Properties of Nonparametric M-Estimation for Mixing Functional Data

Jia Chen,Lixin Zhang

DOI: https://doi.org/10.1016/j.jspi.2008.05.007

IF: 1.095

2009-01-01

Journal of Statistical Planning and Inference

Abstract:We investigate the asymptotic behavior of a nonparametric M-estimator of a regression function for stationary dependent processes, where the explanatory variables take values in some abstract functional space. Under some regularity conditions, we give the weak and strong consistency of the estimator as well as its asymptotic normality. We also give two examples of functional processes that satisfy the mixing conditions assumed in this paper. Furthermore, a simulated example is presented to examine the finite sample performance of the proposed estimator.
A New Recursive Bounded GM Estimator for Regression

Li-hua CHENG,Jun-min ZHANG

DOI: https://doi.org/10.11830/ISSN.1000-5013.2015.03.0359

2015-01-01

Abstract:A new recursive bounded GM estimator for regression is proposed.Unlike other GM estimators,the new esti-mator is based on one more general framework and uses a cost function with bounded M-estimate function.The new esti-mator,in effect,is a recursive one-step iteration solution of the"normal equations"corresponding to the cost function.In the new estimator,a weight function is designed to reject or to reduce the influence of the outliers.Furthermore,by in-troducing an augment variable,the proposed estimator is modified to a very adaptive version for the estimation of autore-gressive parameters.The simulation results show that both the proposed estimator and its modification are more effective than other related estimators in suppressing the adverse influence of outliers;the proposed estimator,with the same set-tings,can keep a high accuracy and stable convergence performance in a variety of non-stationary environments.
Limit Theorems for Dependent Bernoulli Variables with Statistical Inference

Yang Zhang,Lixin Zhang

DOI: https://doi.org/10.1080/03610926.2015.1021015

2017-01-01

Abstract:We consider a class of dependent Bernoulli variables where the conditional success probability is a linear combination of the last few trials and the original success probability. We obtain its limit theorems including the strong law of large numbers, weak invariance principle, and law of the iterated logarithm. We also derive some statistical inference results which make the model applicable. Simulation results are exhibited as well to show that with small sample size the convergence rate is satisfying and the proposed estimators behave well.
Coverage Accuracy of Confidence Intervals in Nonparametric Regression

Song-xi Chen,Yong-song Qin

DOI: https://doi.org/10.1007/s10255-003-0113-3

2003-01-01

Abstract:Point-wise confidence intervals for a nonparametric regression function with random design pointsare considered. The confidence intervals are those based on the traditional normal approximation and theempirical likelihood. Their coverage accuracy is assessed by developing the Edgeworth expansions for thecoverage probabilities. It is shown that the empirical likelihood confidence intervals are Bartlett correctable.
Non-Asymptotic Guarantees for Robust Statistical Learning under Infinite Variance Assumption

Lihu Xu,Fang Yao,Qiuran Yao,Huiming Zhang

2023-01-01

Journal of Machine Learning Research

Abstract:There has been a surge of interest in developing robust estimators for models with heavytailed and bounded variance data in statistics and machine learning, while few works impose unbounded variance. This paper proposes two types of robust estimators, the ridge logtruncated M-estimator and the elastic net log-truncated M-estimator. The first estimator is applied to convex regressions such as quantile regression and generalized linear models, while the other one is applied to high dimensional non-convex learning problems such as robustness of log-truncated estimations over standard estimations.
Smooth Estimation of Error Distribution in Nonparametric Regression under Long Memory

Hira L. Koul,Lihong Wang

DOI: https://doi.org/10.1007/978-3-319-51313-3_5

2017-01-01

Abstract:We consider the problem of estimating the error distribution function in a nonparametric regression model with long memory design and long memory errors. This paper establishes a uniform reduction principle of a smooth weighted residual empirical distribution function estimator. We also investigate consistency property of local Whittle estimator of the long memory parameter based on nonparametric residuals. The results obtained are useful in providing goodness of fit test for the marginal error distribution and in prediction under long memory.
Uniform Consistency of the Nonparametric Error Density Estimation in Regression with Censored Data

XIUQING ZHOU,NINGZHONG SHI,JIN ZHAO

DOI: https://doi.org/10.3969/j.issn.1001-4268.2010.06.007

2010-01-01

Abstract:Nonparametric estimation of the density function of unobservable regression errors is a fundamental issue in regression analysis, because there are many practical applications of error density estimation. This problem for regression models with complete observed data has been studied by several authors. But in many application fields, the corresponding variables are not complete observable because of censoring. In this case, the density function of the unobservable regression errors can be estimated by the kernel type estimator based on the censored regression residuals.In this paper, the asymptotic property of the kernel type estimator is considered and the uniform consistency of the estimator is established.
Simultaneous Confidence Bands in Nonlinear Regression Models with Nonstationarity

Degui Li,Weidong Liu,Qiying Wang,Wei Biao Wu

DOI: https://doi.org/10.5705/ss.202015.0219

IF: 1.4

2017-01-01

Statistica Sinica

Abstract:We consider nonparametric estimation of the regression function g(.) in a nonlinear regression model Y-t = g(X-t)+ sigma(X-t)e(t), where the regressor (X-t) is a nonstationary unit root process and the error (et) is a sequence of independent and identically distributed (i.i.d.) random variables. With proper centering and scaling, the maximum deviation of the local linear estimator of the regression function g is shown to be asymptotically Gumbel. Based on the latter result, we construct simultaneous confidence bands for g, which can be used to test patterns of the regression function. Our results extend existing ones that typically require independent or stationary weakly dependent regressors. We examine the finite sample behavior of the proposed approach via simulated and empirical data examples.
Multivariate Universal Local Linear Kernel Estimators in Nonparametric Regression: Uniform Consistency

Yuliana Linke,Igor Borisov,Pavel Ruzankin,Vladimir Kutsenko,Elena Yarovaya,Svetlana Shalnova

DOI: https://doi.org/10.3390/math12121890

IF: 2.4

2024-06-19

Mathematics

Abstract:In this paper, for a wide class of nonparametric regression models, new local linear kernel estimators are proposed that are uniformly consistent under close-to-minimal and visual conditions on design points. These estimators are universal in the sense that their designs can be either fixed and not necessarily satisfying the traditional regularity conditions, or random, while not necessarily consisting of independent or weakly dependent random variables. With regard to the design elements, only dense filling of the regression function domain with the design points without any specification of their correlation is assumed. This study extends the dense data methodology and main results of the authors' previous work for the case of regression functions of several variables.

mathematics
GLS under monotone heteroskedasticity

Yoichi Arai,Taisuke Otsu,Mengshan Xu

DOI: https://doi.org/10.1016/j.jeconom.2024.105899

IF: 3.363

2024-11-23

Journal of Econometrics

Abstract:The generalized least square (GLS) is one of the most basic tools in regression analyses. A major issue in implementing the GLS is estimation of the conditional variance function of the error term, which typically requires a restrictive functional form assumption for parametric estimation or smoothing parameters for nonparametric estimation. In this paper, we propose an alternative approach to estimate the conditional variance function under nonparametric monotonicity constraints by utilizing the isotonic regression method. Our GLS estimator is shown to be asymptotically equivalent to the infeasible GLS estimator with knowledge of the conditional error variance, and involves only some tuning to trim boundary observations, not only for point estimation but also for interval estimation or hypothesis testing. Simulation studies and an empirical example illustrate excellent finite sample performances of the proposed method.

economics,social sciences, mathematical methods,mathematics, interdisciplinary applications
Posterior consistency of Gaussian process prior for nonparametric binary regression

Subhashis Ghosal,Anindya Roy

DOI: https://doi.org/10.1214/009053606000000795

2007-02-23

Abstract:Consider binary observations whose response probability is an unknown smooth function of a set of covariates. Suppose that a prior on the response probability function is induced by a Gaussian process mapped to the unit interval through a link function. In this paper we study consistency of the resulting posterior distribution. If the covariance kernel has derivatives up to a desired order and the bandwidth parameter of the kernel is allowed to take arbitrarily small values, we show that the posterior distribution is consistent in the $L_1$-distance. As an auxiliary result to our proofs, we show that, under certain conditions, a Gaussian process assigns positive probabilities to the uniform neighborhoods of a continuous function. This result may be of independent interest in the literature for small ball probabilities of Gaussian processes.

Statistics Theory
Simultaneous confidence bands for nonparametric regression with partially missing covariates

Li Cai,Lijie Gu,Qihua Wang,Suojin Wang

DOI: https://doi.org/10.48550/arXiv.2003.00117

2020-02-29

Abstract:In this paper, we consider a weighted local linear estimator based on the inverse selection probability for nonparametric regression with missing covariates at random. The asymptotic distribution of the maximal deviation between the estimator and the true regression function is derived and an asymptotically accurate simultaneous confidence band is constructed. The estimator for the regression function is shown to be oracally efficient in the sense that it is uniformly indistinguishable from that when the selection probabilities are known. Finite sample performance is examined via simulation studies which support our asymptotic theory. The proposed method is demonstrated via an analysis of a data set from the Canada 2010/2011 Youth Student Survey.

Methodology
Kernel estimation of multivariate cumulative distribution function

Rong Liu,Lijian Yang

DOI: https://doi.org/10.1080/10485250802326391

2008-01-01

Journal of Nonparametric Statistics

Abstract:A smooth kernel estimator is proposed for multivariate cumulative distribution functions (cdf), extending the work of Yamato [H. Yamato, Uniform convergence of an estimator of a distribution function, Bull. Math. Statist. 15 (1973), pp. 69-78.] on univariate distribution function estimation. Under assumptions of strict stationarity and geometrically strong mixing, we establish that the proposed estimator follows the same pointwise asymptotically normal distribution of the empirical cdf, while the new estimator is a smooth instead of a step function as the empirical cdf. We also show that under stronger assumptions the smooth kernel estimator has asymptotically smaller mean integrated squared error than the empirical cdf, and converges to the true cdf uniformly almost surely at a rate of (n-1/2log n). Simulated examples are provided to illustrate the theoretical properties. Using the smooth estimator, survival curves for US gross domestic product (GDP) growth are estimated conditional on the unemployment growth rate to examine how GDP growth rate depends on the unemployment policy. Another example of gold and silver price returns is given.
Asymptotic Distributions of Nonparametric Regression Estimators for Longitudinal or Functional Data

Fang Yao

DOI: https://doi.org/10.1016/j.jmva.2006.08.007

IF: 1.387

2006-01-01

Journal of Multivariate Analysis

Abstract:The estimation of a regression function by kernel method for longitudinal or functional data is considered. In the context of longitudinal data analysis, a random function typically represents a subject that is often observed at a small number of time points, while in the studies of functional data the random realization is usually measured on a dense grid. However, essentially the same methods can be applied to both sampling plans, as well as in a number of settings lying between them. In this paper general results are derived for the asymptotic distributions of real-valued functions with arguments which are functionals formed by weighted averages of longitudinal or functional data. Asymptotic distributions for the estimators of the mean and covariance functions obtained from noisy observations with the presence of within-subject correlation are studied. These asymptotic normality results are comparable to those standard rates obtained from independent data, which is illustrated in a simulation study. Besides, this paper discusses the conditions associated with sampling plans, which are required for the validity of local properties of kernel-based estimators for longitudinal or functional data.

Extended Glivenko–Cantelli Theorem in Nonparametric Regression

Asymptotic expansion for nonparametric M-estimator in a nonlinear regression model with long-memory errors

Empirical Likelihood Inference for Nonparametric Regression Functions with Functional Stationary Ergodic Data

Non-parametric Tests for the Tail Equivalence Via Empirical Likelihood

Consistent Specification Testing Via Nonparametric Series Regression

Strong Consistency of the Internal Estimator of Nonparametric Regression with Dependent Data

Strong Limit Theorems for Extended Independent Random Variables and Extended Negatively Dependent Random Variables under Sub-Linear Expectations

Asymptotic Properties of Nonparametric M-Estimation for Mixing Functional Data

A New Recursive Bounded GM Estimator for Regression

Limit Theorems for Dependent Bernoulli Variables with Statistical Inference

Coverage Accuracy of Confidence Intervals in Nonparametric Regression

Non-Asymptotic Guarantees for Robust Statistical Learning under Infinite Variance Assumption

Smooth Estimation of Error Distribution in Nonparametric Regression under Long Memory

Uniform Consistency of the Nonparametric Error Density Estimation in Regression with Censored Data

Simultaneous Confidence Bands in Nonlinear Regression Models with Nonstationarity

Multivariate Universal Local Linear Kernel Estimators in Nonparametric Regression: Uniform Consistency

GLS under monotone heteroskedasticity

Posterior consistency of Gaussian process prior for nonparametric binary regression

Simultaneous confidence bands for nonparametric regression with partially missing covariates

Kernel estimation of multivariate cumulative distribution function

Asymptotic Distributions of Nonparametric Regression Estimators for Longitudinal or Functional Data