Strong Converge of the Recursive M-estimation when Explanatory Variables Are Design Matrices

LIU Dong-hai,MIAO Bai-qi

DOI: https://doi.org/10.3969/j.issn.0253-2778.2006.09.007

2006-01-01

JUSTC

Abstract:The recursive M-estimation algorithm was studied when the explanatory variables were design matrices.Based on some common ideas,the recursive algorithm was proposed and its strong converge was proved.

What problem does this paper attempt to address?

Limiting behavior of recursive M-estimators in multivariate linear regression models and their asymptotic efficiencies

Miao Baiqi,Wu Yuehua,Liu Donghai

DOI: https://doi.org/10.1016/S0252-9602(10)60048-3

2010-01-01

Acta Mathematica Scientia

Abstract:Recursive algorithms are very useful for computing M-estimators of regression coefficients and scatter parameters. In this article, it is shown that for a nondecreasing u1(t), under some mild conditions the recursive M-estimators of regression coefficients and scatter parameters are strongly consistent and the recursive M-estimator of the regression coefficients is also asymptotically normal distributed. Furthermore, optimal recursive M-estimators, asymptotic efficiencies of recursive M-estimators and asymptotic relative efficiencies between recursive M-estimators of regression coefficients are studied.
Limiting Behavior of RecursiveM-Estimators in Multivariate Linear Regression Models

B.Q. Miao,Y. Wu

DOI: https://doi.org/10.1006/jmva.1996.0054

IF: 1.387

1996-01-01

Journal of Multivariate Analysis

Abstract:In this paper, several recursive algorithms for computingM-estimates in multivariate linear regression models are discussed. It is shown that the recursiveM-estimators of regression coefficient and scatter parameters are strongly consistent. In particular, the asymptotic normality of the recursiveM-estimators of regression coefficients is established.
Improve recursive identification using multi-iteration

Xiangping Zhang,Yucai Zhu,Paul P J Van Den Bosch

DOI: https://doi.org/10.3182/20090706-3-FR-2004.00073

2009-01-01

IFAC Proceedings Volumes

Abstract:A multi-iteration recursive identification algorithm is proposed and studied. The algorithm performs multiple iterations for each data sample in order to improve the accuracy of the estimate. The algorithm is worked out for an ARMAX model using Gauss Newton method and pseudo-linear regression. The convergence of the methods is proved by using stochastic process theory and martingale convergence technique. Simulation results show that the algorithm has higher accuracy in parameter estimation and faster convergence than the conventional recursive method.
Selecting an Adaptive Sequence for Computing Recursive M-estimators in Multivariate Linear Regression Models

Baiqi Miao,Qian Tong,Yuehua Wu,Baisuo Jin

DOI: https://doi.org/10.1007/s11424-013-1113-x

2013-01-01

Abstract:In this paper, the authors consider an adaptive recursive algorithm by selecting an adaptive sequence for computing M-estimators in multivariate linear regression models. Its asymptotic property is investigated. The recursive algorithm given by Miao and Wu (1996) is modified accordingly. Simulation studies of the algorithm is also provided. In addition, the Newton-Raphson iterative algorithm is considered for the purpose of comparison.
ROBUST ESTIMATION IN PARAMETRIC TIME SERIES MODELS UNDER LONG‐ AND SHORT‐RANGE‐DEPENDENT STRUCTURES

Jiti Gao,Degui Li,Zhengyan Lin

DOI: https://doi.org/10.1111/j.1467-842x.2009.00537.x

2009-01-01

Australian & New Zealand Journal of Statistics

Abstract:SummaryThis paper studies the asymptotic behaviour of an M‐estimator of regression parameters in the linear model when the design variables are either stationary short‐range dependent (SRD), α‐mixing or long‐range dependent (LRD), and the errors are LRD. The weak consistency and the asymptotic distributions of the M‐estimator are established. We present some simulated examples to illustrate the efficiency of the proposed M‐estimation method.
Strong Convergence Theorems for Relatively Nonexpansive Mappings and Variational Inclusions in Banach Spaces

LIU Ying,CHEN Yong-li,WANG Xian

2008-01-01

Abstract:In this paper,we consider an iterative scheme for finding a common element of the set of fixed points of a relatively nonexpansive mapping and the set of solutions of the variational inclusion for an inverse-strongly accretive mapping and m-accretive mapping in a real Banach space.Then we show that the sequence converges strongly to a common element of two sets.
A New Recursive Bounded GM Estimator for Regression

Li-hua CHENG,Jun-min ZHANG

DOI: https://doi.org/10.11830/ISSN.1000-5013.2015.03.0359

2015-01-01

Abstract:A new recursive bounded GM estimator for regression is proposed.Unlike other GM estimators,the new esti-mator is based on one more general framework and uses a cost function with bounded M-estimate function.The new esti-mator,in effect,is a recursive one-step iteration solution of the"normal equations"corresponding to the cost function.In the new estimator,a weight function is designed to reject or to reduce the influence of the outliers.Furthermore,by in-troducing an augment variable,the proposed estimator is modified to a very adaptive version for the estimation of autore-gressive parameters.The simulation results show that both the proposed estimator and its modification are more effective than other related estimators in suppressing the adverse influence of outliers;the proposed estimator,with the same set-tings,can keep a high accuracy and stable convergence performance in a variety of non-stationary environments.
Asymptotic Normality of the Recursive M-estimators of the Scale Parameters

Baiqi Miao,Yuehua Wu,Donghai Liu,Qian Tong

DOI: https://doi.org/10.1007/s10463-006-0051-y

2006-01-01

Annals of the Institute of Statistical Mathematics

Abstract:In this paper, the limit distributions of the recursive M-estimators of scatter parameters in a multivariate linear model setting are studied. Under some mild conditions, the asymptotic normality of the recursive M-esimtators is established. Some Monte Carlo simulation results are presented to illustrate the performance of the recursive M-estimators.
The Estimate Of The Rank For Regression Coefficient Matrix In A Median Regression-Model

Bq Miao

1992-01-01

Chinese Annals of Mathematics Series B

Abstract:It is discussed to infer the rank of regression coefficient matrix in a multivariate linear regression model. If the zero median vector is unique and the design matrices satisfy some weaker conditions, it is derived that the estimators of the rank of regression coefficient matrix under the minimum distance criterion by using model selection method is strongly consistent.
Limiting Behavior of Recursive Algorithm for Estimates of Parameters in Single Index Models

戴静,缪柏其,金百锁

2009-01-01

Abstract:Single index model in semi-parametric statistics was studied,and the recursive algorithm(used in large samples) in single index model was analyzed.The strong consistency and the asymptotic normality of the estimators were established.Finally,the limiting behavior was proposed.
Local linear M-estimation for spatial processes in fixed-design models

Jia Chen,Li-Xin Zhang

DOI: https://doi.org/10.1007/s00184-009-0233-8

IF: 0.96

2009-01-01

Metrika

Abstract:We investigate the local linear M-estimation for regression in a fixed-design model when the errors are from a strongly mixing random field. We establish the weak and strong consistency as well as the asymptotic normality of the local linear M-estimator. The conditions on ρ (·) used in this paper are mild and allow many important special cases such as the least square estimator and the least absolute distance estimator.
Strong Approximation Theorems for Sums of Random Variables when Extreme Terms Are Excluded

Li Xin Zhang

DOI: https://doi.org/10.1007/s101140200162

2002-01-01

Acta Mathematica Sinica English Series

Abstract:Let {X n ;n≥1} be a sequence of i.i.d. random variables and let \(X^{{{\left( r \right)}}}_{n} = X_{j} \) if |X j | is the r-th maximum of |X 1|, ..., |X n |. Let S n = X 1+⋯+X n and \({}^{{{\left( r \right)}}}S_{n} = S_{n} - {\left( {X^{{{\left( 1 \right)}}}_{n} + \cdots + X^{{{\left( r \right)}}}_{n} } \right)}.\)Sufficient and necessary conditions for (r) S n approximating to sums of independent normal random variables are obtained. Via approximation results, the convergence rates of the strong law of large numbers for (r) S n are studied.
Robust Kernel Recursive Adaptive Filtering Algorithms Based on M-estimate

Xinyue Yang,Yifan Mu,Kui Cao,Mengzhuo Lv,Bei Peng,Ying Zhang,Gang Wang

DOI: https://doi.org/10.1016/j.sigpro.2023.108952

IF: 4.729

2023-01-01

Signal Processing

Abstract:When coping with the large outliers in measurement caused by the non-Gaussian environmental noise, although the MCC criterion adopts the high-order statistics, the residual error for large outliers still exists. Considering that the M-estimate works well in minimum square error criterion and it can truncate the outliers and further improve the robustness, in this paper, we propose the robust kernel recursive least squares algorithms and the robust kernel recursive maximum correntropy algorithms based on three M-estimate methods. Then, numerical simulations verify that the M-estimates help the proposed algorithms have better performance than the conventional kernel recursive adaptive filtering against the non-Gaussian noise.
Strong Convergence Rate for Slope Change Point Estimator

HUI Jun,TAN Changchun,MIAO Baiqi

DOI: https://doi.org/10.3969/j.issn.0253-2778.2010.09.005

2010-01-01

JUSTC

Abstract:The problem of slope-change point was discussed.The consistency and convergence rate of the change point estimator were presented.At the same time,the convergence rate of the change point estimator was obtained when the magnitude of the slope change was small.
Convergence of Inexact Steepest Descent Algorithm for Multiobjective Optimizations on Riemannian Manifolds Without Curvature Constraints

X. M. Wang,J. H. Wang,C. Li

DOI: https://doi.org/10.1007/s10957-023-02235-y

2023-01-01

Abstract:We study the issue of convergence for inexact steepest descent algorithm (employing general step sizes) for multiobjective optimizations on general Riemannian manifolds (without curvature constraints). Under the assumption of the local convexity/quasi-convexity, local/global convergence results are established. Furthermore, without the assumption of the local convexity/quasi-convexity, but under an error bound-like condition, local/global convergence results and convergence rate estimates are presented, which are new even in the linear space setting. Our results improve/extend the corresponding ones in (Wang et al. in SIAM J Optim 31(1):172–199, 2021) for scalar optimization problems on Riemannian manifolds to multiobjective ones. Finally, for the special case when the inexact steepest descent algorithm employing Armijo rule, our results improve/extend the corresponding ones in (Ferreira et al. in J Optim Theory Appl 184:507–533, 2020) by relaxing curvature constraints.
Recursive estimation in large panel data models: Theory and practice

Bin Jiang,Yanrong Yang,Jiti Gao,Cheng Hsiao

DOI: https://doi.org/10.1016/j.jeconom.2020.07.055

IF: 3.363

2021-10-01

Journal of Econometrics

Abstract:<p>Bai (2009) proposes recursive estimation for panel data models with interactive effects. We study the behaviours of this recursive estimator. The recursive formula is established that shows the behaviours of recursive estimators depend on the initial estimator, the population structure and the iterative steps. Under some general scenarios, we find that the recursive estimator becomes consistent after the first iteration from any initials. We also obtain the optimal number of iterative steps under some prescribed conditions. The central limit theorem of the recursive estimator is established when the initial estimator is OLS. Various simulations are conducted to support our theoretical findings.</p>

economics,social sciences, mathematical methods,mathematics, interdisciplinary applications
A Mixed Evolutionary Algorithm to Solve the O-D Matrix Estimation Problem

XY Du,P Li,GY Ma,Y Wen

DOI: https://doi.org/10.1109/icnnb.2005.1614572

2005-01-01

Abstract:The entropy maximizing (EM) model is a main approach to solve the O-D matrix estimation problem, but the current algorithms solving the EM model is limited by the difficulty of choosing a proper initial solution for searching procedure, such as the Newton's method and the Levenberg-Marquardt algorithm (LMA). A mixed evolutionary algorithm (MEA) integrating genetic algorithm (GA) and LMA is presented here to solve the EM model. In MEA, GA is used to obtain a solution which is in the neighbourhood of the optimal solution which satisfies the precision first. Then considering that GA is slow to converge, LMA is applied to quickly converge into the optimal solution which satisfies the precision with GA's result as its initial solution. The numerical results show that MEA performs quite better than GA and LMA in solving the EM model
Strong convergence rate of estimators of change point and its application

Xiaoping Shi,Yuehua Wu,Baiqi Miao

DOI: https://doi.org/10.1016/j.csda.2008.11.015

IF: 2.035

2009-01-01

Computational Statistics & Data Analysis

Abstract:Let {Xn,n⩾1} be an independent sequence with a mean shift. We consider the cumulative sum (CUSUM) estimator of a change point. It is shown that, when the rth moment of Xn is finite, for n⩾1 and r>1, strong convergence rate of the change point estimator is o(M(n)/n), for any M(n) satisfying that M(n)↑∞, which has improved the results in the literature. Furthermore, it is also shown that the preceding rate is still valid for some dependent or negative associate cases. We also propose an iterative algorithm to search for the location of a change point. A simulation study on a mean shift model with a stable distribution is provided, which demonstrates that the algorithm is efficient. In addition, a real data example is given for illustration.
M-estimation for linear models with exchangeable errors

Li Wang,Weidong Ma,Ying Yang

DOI: https://doi.org/10.1080/02331888.2024.2432558

IF: 2.346

2024-12-01

Statistics

Abstract:This paper focuses on the estimation of parameters in linear models with fixed regressors and exchangeable errors, especially, these errors are assumed to be a subset of an infinite exchangeable sequence. We propose a modified M-estimation to estimate the slope parameter, which outperforms the traditional M-estimation when the random errors are exchangeable. The modified M-estimation utilizes a design point centralization technique, leading to a more suitable estimator with improved theoretical properties. To establish the asymptotic properties of the modified M-estimator, we also develop a weighted central limit theorem for exchangeable sequences, which is of independent interest. Regarding the intercept, we prove that there is no consistent estimator for the intercept when the exchangeable errors have non-zero covariance, and we propose an unbiased estimator for the intercept as a viable compromise. Based on our theoretical findings, we develop a simulation-based approach to approximate the confidence set of the slope, and we establish a prediction model specifically designed for linear models with exchangeable errors. Additionally, we illustrate the finite sample performance of the proposed method through various numerical simulations and a real data application.

statistics & probability
Identifiability Implies Robust, Globally Exponentially Convergent On-line Parameter Estimation: Application to Model Reference Adaptive Control

Lei Wang,Romeo Ortega,Alexey Bobtsov,Jose Guadalupe Romero,Bowen Yi

DOI: https://doi.org/10.1080/00207179.2023.2246595

IF: 2.102

2024-01-01

International Journal of Control

Abstract:In this paper we propose a new parameter estimator that ensures global exponential convergence of linear regression models requiring only the necessary assumption of identifiability of the regression equation, which we show is equivalent to interval excitation of the regressor vector. Continuous and discrete-time versions of the estimators are given. An extension to—separable and monotonic—non-linear parameterizations is also given. The estimators are shown to be robust to additive measurement noise and—not necessarily slow—parameter variations. Moreover, a version of the continuous-time estimator that rejects sinusoidal disturbances with unknown internal model is given. The estimator is shown to be applicable to the classical model reference adaptive control problem relaxing the conspicuous assumption of known sign of the high-frequency gain. Simulation results that illustrate the performance of the estimator are given.

Strong Converge of the Recursive M-estimation when Explanatory Variables Are Design Matrices

Limiting behavior of recursive M-estimators in multivariate linear regression models and their asymptotic efficiencies

Limiting Behavior of RecursiveM-Estimators in Multivariate Linear Regression Models

Improve recursive identification using multi-iteration

Selecting an Adaptive Sequence for Computing Recursive M-estimators in Multivariate Linear Regression Models

ROBUST ESTIMATION IN PARAMETRIC TIME SERIES MODELS UNDER LONG‐ AND SHORT‐RANGE‐DEPENDENT STRUCTURES

Strong Convergence Theorems for Relatively Nonexpansive Mappings and Variational Inclusions in Banach Spaces

A New Recursive Bounded GM Estimator for Regression

Asymptotic Normality of the Recursive M-estimators of the Scale Parameters

The Estimate Of The Rank For Regression Coefficient Matrix In A Median Regression-Model

Limiting Behavior of Recursive Algorithm for Estimates of Parameters in Single Index Models

Local linear M-estimation for spatial processes in fixed-design models

Strong Approximation Theorems for Sums of Random Variables when Extreme Terms Are Excluded

Robust Kernel Recursive Adaptive Filtering Algorithms Based on M-estimate

Strong Convergence Rate for Slope Change Point Estimator

Convergence of Inexact Steepest Descent Algorithm for Multiobjective Optimizations on Riemannian Manifolds Without Curvature Constraints

Recursive estimation in large panel data models: Theory and practice

A Mixed Evolutionary Algorithm to Solve the O-D Matrix Estimation Problem

Strong convergence rate of estimators of change point and its application

M-estimation for linear models with exchangeable errors

Identifiability Implies Robust, Globally Exponentially Convergent On-line Parameter Estimation: Application to Model Reference Adaptive Control