Improved estimation via model selection method for semimartingale regressions based on discrete data

Evgeny A. Pchelintsev,Serguei M. Pergamenshchikov,Maria A. Povzun

DOI: https://doi.org/10.48550/arXiv.1909.07232

2019-09-16

Statistics Theory

Abstract:We consider the robust adaptive nonparametric estimation problem for a periodic function observed in the framework of a continuous time regression model with semimartingale noises.

What problem does this paper attempt to address?

Adaptive model selection method for a conditionally Gaussian semimartingale regression in continuous time

Evgeny Pchelintsev,Serguei Pergamenshchikov

DOI: https://doi.org/10.17223/19988621/58/2

2018-11-12

Abstract:This paper considers the problem of robust adaptive efficient estimating of a periodic function in a continuous time regression model with the dependent noises given by a general square integrable semimartingale with a conditionally Gaussian distribution. An example of such noise is the non-Gaussian Ornstein-Uhlenbeck-Levy processes. An adaptive model selection procedure, based on the improved weighted least square estimates, is proposed. Under some conditions on the noise distribution, sharp oracle inequality for the robust risk has been proved and the robust efficiency of the model selection procedure has been established. The numerical analysis results are given.

Statistics Theory
Regularization and variable selection for infinite variance autoregressive models

Ganggang Xu,Yanbiao Xiang,Suojin Wang,Zhengyan Lin

DOI: https://doi.org/10.1016/j.jspi.2012.03.014

IF: 1.095

2012-01-01

Journal of Statistical Planning and Inference

Abstract:Autoregressive models with infinite variance are of great importance in modeling heavy-tailed time series and have been well studied. In this paper, we propose a penalized method to conduct model selection for autoregressive models with innovations having Pareto-like distributions with index α∈(0,2). By combining the least absolute deviation loss function and the adaptive lasso penalty, the proposed method is able to consistently identify the true model and at the same time produce efficient estimators with a convergence rate of n−1/α. In addition, our approach provides a unified way to conduct variable selection for autoregressive models with finite or infinite variance. A simulation study and a real data analysis are conducted to illustrate the effectiveness of our method.
Improved estimation in a non-Gaussian parametric regression

Evgeny Pchelintsev

DOI: https://doi.org/10.1007/s11203-013-9075-0

2011-09-29

Abstract:The paper considers the problem of estimating the parameters in a continuous time regression model with a non-Gaussian noise of pulse type. The noise is specified by the Ornstein-Uhlenbeck process driven by the mixture of a Brownian motion and a compound Poisson process. Improved estimates for the unknown regression parameters, based on a special modification of the James-Stein procedure with smaller quadratic risk than the usual least squares estimates, are proposed. The developed estimation scheme is applied for the improved parameter estimation in the discrete time regression with the autoregressive noise depending on unknown nuisance parameters.

Statistics Theory
Rate-optimal estimation of mixed semimartingales

Carsten H. Chong,Thomas Delerue,Fabian Mies

2024-10-25

Abstract:Consider the sum $Y=B+B(H)$ of a Brownian motion $B$ and an independent fractional Brownian motion $B(H)$ with Hurst parameter $H\in(0,1)$. Even though $B(H)$ is not a semimartingale, it was shown in [\textit{Bernoulli} \textbf{7} (2001) 913--934] that $Y$ is a semimartingale if $H>3/4$. Moreover, $Y$ is locally equivalent to $B$ in this case, so $H$ cannot be consistently estimated from local observations of $Y$. This paper pivots on another unexpected feature in this model: if $B$ and $B(H)$ become correlated, then $Y$ will never be a semimartingale, and $H$ can be identified, regardless of its value. This and other results will follow from a detailed statistical analysis of a more general class of processes called \emph{mixed semimartingales}, which are semiparametric extensions of $Y$ with stochastic volatility in both the martingale and the fractional component. In particular, we derive consistent estimators and feasible central limit theorems for all parameters and processes that can be identified from high-frequency observations. We further show that our estimators achieve optimal rates in a minimax sense.

Statistics Theory,Probability,Methodology
Robust Variable Selection Via Nonconcave Penalties with an Upgraded Parsimonious Dynamic Covariance Modeling

Mengfei Ran,Lin Xu

DOI: https://doi.org/10.1080/03610918.2022.2155308

2024-01-01

Communications in Statistics - Simulation and Computation

Abstract:We present a new parsimonious method for joint mean-covariance modeling based on M-estimation and nonconcave penalty. In this paper, the robustness of the proposed model was aimed at addressing the issue when the working matrix is misspecified and a spot of outliers exist in the dataset. The proposed approach outperforms the traditional method in robustness and variable selections for longitudinal data analysis, particularly when the dataset contains a spot of outliers. The simulation results back up the theoretical findings, and the methodology is further illustrated via an analysis of a real progesterone data example.
PENALIZED VARIABLE SELECTION PROCEDURE FOR COX MODELS WITH SEMIPARAMETRIC RELATIVE RISK

Pang Du,Shuangge Ma,Hua Liang

DOI: https://doi.org/10.1214/09-AOS780

2010-08-01

Abstract:We study the Cox models with semiparametric relative risk, which can be partially linear with one nonparametric component, or multiple additive or nonadditive nonparametric components. A penalized partial likelihood procedure is proposed to simultaneously estimate the parameters and select variables for both the parametric and the nonparametric parts. Two penalties are applied sequentially. The first penalty, governing the smoothness of the multivariate nonlinear covariate effect function, provides a smoothing spline ANOVA framework that is exploited to derive an empirical model selection tool for the nonparametric part. The second penalty, either the smoothly-clipped-absolute-deviation (SCAD) penalty or the adaptive LASSO penalty, achieves variable selection in the parametric part. We show that the resulting estimator of the parametric part possesses the oracle property, and that the estimator of the nonparametric part achieves the optimal rate of convergence. The proposed procedures are shown to work well in simulation experiments, and then applied to a real data example on sexually transmitted diseases.
Simultaneous sparse model selection and coefficient estimation for heavy-tailed autoregressive processes

Hailin Sang,Yan Sun

DOI: https://doi.org/10.48550/arXiv.1112.2682

2013-09-21

Abstract:We propose a sparse coefficient estimation and automated model selection procedure for autoregressive (AR) processes with heavy-tailed innovations based on penalized conditional maximum likelihood. Under mild moment conditions on the innovation processes, the penalized conditional maximum likelihood estimator (PCMLE) satisfies a strong consistency, $O_P(N^{-1/2})$ consistency, and the oracle properties, where N is the sample size. We have the freedom in choosing penalty functions based on the weak conditions on them. Two penalty functions, least absolute shrinkage and selection operator (LASSO) and smoothly clipped average deviation (SCAD), are compared. The proposed method provides a distribution-based penalized inference to AR models, which is especially useful when the other estimation methods fail or under perform for AR processes with heavy-tailed innovations (see \cite{Resnick}). A simulation study confirms our theoretical results. At the end, we apply our method to a historical price data of the US Industrial Production Index for consumer goods, and obtain very promising results.

Methodology,Statistics Theory
Adaptive Covariance Estimation with model selection

Rolando Biscay,Hélène Lescornel,Jean-Michel Loubes

DOI: https://doi.org/10.48550/arXiv.1203.0107

2012-03-01

Abstract:We provide in this paper a fully adaptive penalized procedure to select a covariance among a collection of models observing i.i.d replications of the process at fixed observation points. For this we generalize previous results of Bigot and al. and propose to use a data driven penalty to obtain an oracle inequality for the estimator. We prove that this method is an extension to the matricial regression model of the work by Baraud.

Statistics Theory
Distributed Estimation and Inference for Semi-parametric Binary Response Models

Xi Chen,Wenbo Jing,Weidong Liu,Yichen Zhang

2024-08-16

Abstract:The development of modern technology has enabled data collection of unprecedented size, which poses new challenges to many statistical estimation and inference problems. This paper studies the maximum score estimator of a semi-parametric binary choice model under a distributed computing environment without pre-specifying the noise distribution. An intuitive divide-and-conquer estimator is computationally expensive and restricted by a non-regular constraint on the number of machines, due to the highly non-smooth nature of the objective function. We propose (1) a one-shot divide-and-conquer estimator after smoothing the objective to relax the constraint, and (2) a multi-round estimator to completely remove the constraint via iterative smoothing. We specify an adaptive choice of kernel smoother with a sequentially shrinking bandwidth to achieve the superlinear improvement of the optimization error over the multiple iterations. The improved statistical accuracy per iteration is derived, and a quadratic convergence up to the optimal statistical error rate is established. We further provide two generalizations to handle the heterogeneity of datasets and high-dimensional problems where the parameter of interest is sparse.

Statistics Theory,Machine Learning
A robust and efficient estimation and variable selection method for partially linear models with large-dimensional covariates

Hu Yang,Ning Li,Jing Yang

DOI: https://doi.org/10.1007/s00362-018-1013-1

2018-05-21

Statistical Papers

Abstract:In this paper, a new robust and efficient estimation approach based on local modal regression is proposed for partially linear models with large-dimensional covariates. We show that the resulting estimators for both parametric and nonparametric components are more efficient in the presence of outliers or heavy-tail error distribution, and as asymptotically efficient as the corresponding least squares estimators when there are no outliers and the error distribution is normal. We also establish the asymptotic properties of proposed estimators when the covariate dimension diverges at the rate of on\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$o\left( {\sqrt{n} } \right) \mathrm{{ }}$$\end{document}. To achieve sparsity and enhance interpretability, we develop a variable selection procedure based on SCAD penalty to select significant parametric covariates and show that the method enjoys the oracle property under mild regularity conditions. Moreover, we propose a practical modified MEM algorithm for the proposed procedures. Some Monte Carlo simulations and a real data are conducted to illustrate the finite sample performance of the proposed estimators. Finally, based on the idea of sure independence screening procedure proposed by Fan and Lv (J R Stat Soc 70:849–911, 2008), a robust two-step approach is introduced to deal with ultra-high dimensional data.

statistics & probability
Robust variable selection for partially linear additive models

Graciela Boente,Alejandra Martínez

2024-01-31

Abstract:Among semiparametric regression models, partially linear additive models provide a useful tool to include additive nonparametric components as well as a parametric component, when explaining the relationship between the response and a set of explanatory variables. This paper concerns such models under sparsity assumptions for the covariates included in the linear component. Sparse covariates are frequent in regression problems where the task of variable selection is usually of interest. As in other settings, outliers either in the residuals or in the covariates involved in the linear component have a harmful effect. To simultaneously achieve model selection for the parametric component of the model and resistance to outliers, we combine preliminary robust estimators of the additive component, robust linear $MM-$regression estimators with a penalty such as SCAD on the coefficients in the parametric part. Under mild assumptions, consistency results and rates of convergence for the proposed estimators are derived. A Monte Carlo study is carried out to compare, under different models and contamination schemes, the performance of the robust proposal with its classical counterpart. The obtained results show the advantage of using the robust approach. Through the analysis of a real data set, we also illustrate the benefits of the proposed procedure.

Methodology,Statistics Theory
Doubly Robust Semiparametric Inference Using Regularized Calibrated Estimation with High-dimensional Data

Satyajit Ghosh,Zhiqiang Tan

DOI: https://doi.org/10.48550/arXiv.2009.12033

2020-09-25

Abstract:Consider semiparametric estimation where a doubly robust estimating function for a low-dimensional parameter is available, depending on two working models. With high-dimensional data, we develop regularized calibrated estimation as a general method for estimating the parameters in the two working models, such that valid Wald confidence intervals can be obtained for the parameter of interest under suitable sparsity conditions if either of the two working models is correctly specified. We propose a computationally tractable two-step algorithm and provide rigorous theoretical analysis which justifies sufficiently fast rates of convergence for the regularized calibrated estimators in spite of sequential construction and establishes a desired asymptotic expansion for the doubly robust estimator. As concrete examples, we discuss applications to partially linear, log-linear, and logistic models and estimation of average treatment effects. Numerical studies in the former three examples demonstrate superior performance of our method, compared with debiased Lasso.

Methodology
Variable selection in semiparametric regression modeling

Runze Li,Hua Liang

DOI: https://doi.org/10.48550/arXiv.0803.1931

2008-03-13

Statistics Theory

Abstract:In this paper, we are concerned with how to select significant variables in semiparametric modeling. Variable selection for semiparametric regression models consists of two components: model selection for nonparametric components and selection of significant variables for the parametric portion. Thus, semiparametric variable selection is much more challenging than parametric variable selection (e.g., linear and generalized linear models) because traditional variable selection procedures including stepwise regression and the best subset selection now require separate model selection for the nonparametric components for each submodel. This leads to a very heavy computational burden. In this paper, we propose a class of variable selection procedures for semiparametric regression models using nonconcave penalized likelihood. We establish the rate of convergence of the resulting estimate. With proper choices of penalty functions and regularization parameters, we show the asymptotic normality of the resulting estimate and further demonstrate that the proposed procedures perform as well as an oracle procedure. A semiparametric generalized likelihood ratio test is proposed to select significant variables in the nonparametric component. We investigate the asymptotic behavior of the proposed test and demonstrate that its limiting null distribution follows a chi-square distribution which is independent of the nuisance parameters. Extensive Monte Carlo simulation studies are conducted to examine the finite sample performance of the proposed variable selection procedures.
On improved estimation in a conditionally Gaussian regression

Evgeny Pchelintsev

DOI: https://doi.org/10.48550/arXiv.1105.5036

2011-05-25

Abstract:The paper considers the problem of estimating a $p\geq2$\ dimensional mean vector of a multivariate conditionally normal distribution under quadratic loss. The problem of this type arises when estimating the parameters in a continuous time regression model with a non-Gaussian Ornstein--Uhlenbeck process driven by the mixture of a Brownian motion and a compound Poisson process.

Statistics Theory
Automatic variable selection for semiparametric spatial autoregressive model

Fang Lu,Sisheng Liu,Jing Yang,Xuewen Lu

DOI: https://doi.org/10.1080/07474938.2023.2225947

IF: 1.2

2023-07-14

Econometric Reviews

Abstract:This article studies the generalized method of moment estimation of semiparametric varying coefficient partially linear spatial autoregressive model. The technique of profile least squares is employed and all estimators have explicit formulas which are computationally convenient. We derive the limiting distributions of the proposed estimators for both parametric and non parametric components. Variable selection procedures based on smooth-threshold estimating equations are proposed to automatically eliminate irrelevant parameters and zero varying coefficient functions. Compared to the alternative approaches based on shrinkage penalty, the new method is easily implemented. Oracle properties of the resulting estimators are established. Large amounts of Monte Carlo simulations confirm our theories and demonstrate that the estimators perform reasonably well in finite samples. We also apply the novel methods to an empirical data analysis.

economics,social sciences, mathematical methods,mathematics, interdisciplinary applications,statistics & probability
Semiparametric modeling of autonomous nonlinear dynamical systems with applications

Debashis Paul,Jie Peng,Prabir Burman

DOI: https://doi.org/10.48550/arXiv.0906.3501

2009-06-19

Abstract:In this paper, we propose a semi-parametric model for autonomous nonlinear dynamical systems and devise an estimation procedure for model fitting. This model incorporates subject-specific effects and can be viewed as a nonlinear semi-parametric mixed effects model. We also propose a computationally efficient model selection procedure. We prove consistency of the proposed estimator under suitable regularity conditions. We show by simulation studies that the proposed estimation as well as model selection procedures can efficiently handle sparse and noisy measurements. Finally, we apply the proposed method to a plant growth data used to study growth displacement rates within meristems of maize roots under two different experimental conditions.

Methodology,Statistics Theory
Globally consistent model selection in semi-parametric additive coefficient models

shuping jiang,lan xue

DOI: https://doi.org/10.1080/10485252.2015.1083566

2015-01-01

Journal of Nonparametric Statistics

Abstract:We study a penalised polynomial spline (PPS) method for model selection in additive coefficient models. It approximates nonparametric coefficient functions by polynomial splines and minimises the sum of squared errors subject to an additive penalty on the norms of spline functions. For non-convex penalty functions such as smoothly clipped absolute deviation (SCAD) penalty, we investigate the asymptotic properties of the global solution of the non-convex objective function. We establish explicitly that the oracle estimator is the global solution with probability approaching one. Therefore, the global solution enjoys both model estimation and selection consistency. In the literature, the asymptotic properties of local solutions rather than global solutions are well-established for non-convex penalty functions. Our theoretical results broaden the traditional understanding of the PPS method. Extensive Monte Carlo simulation studies show the proposed method performs well numerically. We also illustrate the use of the proposed method by analysing a housing price data set.
Efficient estimation of parameters in marginals in semiparametric multivariate models

Ivan Medovikov,Valentyn Panchenko,Artem Prokhorov

2024-01-30

Abstract:We consider a general multivariate model where univariate marginal distributions are known up to a parameter vector and we are interested in estimating that parameter vector without specifying the joint distribution, except for the marginals. If we assume independence between the marginals and maximize the resulting quasi-likelihood, we obtain a consistent but inefficient QMLE estimator. If we assume a parametric copula (other than independence) we obtain a full MLE, which is efficient but only under a correct copula specification and may be biased if the copula is misspecified. Instead we propose a sieve MLE estimator (SMLE) which improves over QMLE but does not have the drawbacks of full MLE. We model the unknown part of the joint distribution using the Bernstein-Kantorovich polynomial copula and assess the resulting improvement over QMLE and over misspecified FMLE in terms of relative efficiency and robustness. We derive the asymptotic distribution of the new estimator and show that it reaches the relevant semiparametric efficiency bound. Simulations suggest that the sieve MLE can be almost as efficient as FMLE relative to QMLE provided there is enough dependence between the marginals. We demonstrate practical value of the new estimator with several applications. First, we apply SMLE in an insurance context where we build a flexible semi-parametric claim loss model for a scenario where one of the variables is censored. As in simulations, the use of SMLE leads to tighter parameter estimates. Next, we consider financial risk management examples and show how the use of SMLE leads to superior Value-at-Risk predictions. The paper comes with an online archive which contains all codes and datasets.

General Economics,Statistics Theory,Applications,Methodology
Semiparametric least squares estimation of binary choice panel data models with endogeneity

Anastasia Semykina,Yimeng Xie,Cynthia Fan Yang,Qiankun Zhou

DOI: https://doi.org/10.1016/j.econmod.2024.106661

IF: 4.7

2024-01-27

Economic Modelling

Abstract:This paper investigates the estimation of binary response panel data models with endogenous regressors using semiparametric least squares. The endogeneity arises from an unobserved time-invariant effect and a nonzero correlation of between the idiosyncratic error and one or more explanatory variables. Our proposed estimator addresses endogeneity by the correlated random effects and control function approaches. The estimator is shown to be asymptotically normally distributed and to have satisfactory finite sample properties in Monte Carlo experiments. The method's utility is demonstrated through an empirical application that examines the effect of a husband's income on the labor-force participation of married women.

economics

Improved estimation via model selection method for semimartingale regressions based on discrete data

Adaptive model selection method for a conditionally Gaussian semimartingale regression in continuous time

Regularization and variable selection for infinite variance autoregressive models

Improved estimation in a non-Gaussian parametric regression

Rate-optimal estimation of mixed semimartingales

Robust Variable Selection Via Nonconcave Penalties with an Upgraded Parsimonious Dynamic Covariance Modeling

PENALIZED VARIABLE SELECTION PROCEDURE FOR COX MODELS WITH SEMIPARAMETRIC RELATIVE RISK

Simultaneous sparse model selection and coefficient estimation for heavy-tailed autoregressive processes

Adaptive Covariance Estimation with model selection

Distributed Estimation and Inference for Semi-parametric Binary Response Models

A robust and efficient estimation and variable selection method for partially linear models with large-dimensional covariates

Robust variable selection for partially linear additive models

Doubly Robust Semiparametric Inference Using Regularized Calibrated Estimation with High-dimensional Data

Variable selection in semiparametric regression modeling

On improved estimation in a conditionally Gaussian regression

Automatic variable selection for semiparametric spatial autoregressive model

Semiparametric modeling of autonomous nonlinear dynamical systems with applications

Globally consistent model selection in semi-parametric additive coefficient models

Efficient estimation of parameters in marginals in semiparametric multivariate models

Semiparametric least squares estimation of binary choice panel data models with endogeneity