The projection algorithm for estimation of nonlinear system parameter

Zhong-sheng HOU,Bai-sheng YU,Wen-hu HUANG

DOI: https://doi.org/10.3321/j.issn:0367-6234.2000.03.007

2000-01-01

Abstract:Studies estimation of time-varying parameters for nonlinear systems, presents a projection algorithm based on a new estimation criterion improved gradient, and the proof of its convergence, and illustrates its effectiveness and advantage with an example

What problem does this paper attempt to address?

Identification of nonlinearly parameterized nonlinear models: application to mass balance systems.

Xiangbin Liu,Romeo Ortega,Hongye Su,Jian Chu

DOI: https://doi.org/10.1109/CDC.2009.5399817

2009-01-01

Abstract:A new framework to design parameter estimators for nonlinearly parameterized systems is proposed in this paper. The key step is the construction of a monotone function, which explicitly depends on some of the estimator tuning parameters. Monotonicity-or the related property of convexity-have already been explored by several authors with monotonicity (or convexity) being a priori assumptions that are, usually, valid only on some region of state space. In our approach monotonicity is enforced by the designer, effectively becoming a synthesis tool. In order to dispose of degrees of freedom to render the function monotone we depart from standard (gradient or least-squares) estimators and adopt instead the recently introduced immersion and invariance approach for adaptation.
Parameter Estimation in Exponential Models by Linear and Nonlinear Fitting Methods.

Ping Yang,Chao-peng Wu,Yi-lu Guo,Hong-bo Liu,Hui Huang,Hang-zhou Wang,Shu-yue Zhan,Bang-yi Tao,Quan-quan Mu,Qiang Wang,Hong Song

DOI: https://doi.org/10.1631/fitee.1601683

IF: 2.526

2017-01-01

Frontiers of Information Technology & Electronic Engineering

Abstract:Estimation of unknown parameters in exponential models by linear and nonlinear fitting methods is discussed. Based on the extreme value theorem and Taylor series expansion, it is proved theoretically that the parameters estimated by the linear fitting method alone cannot minimize the sum of the squared residual errors in the measurement data when measurement noise is involved in the data. Numerical simulation is performed to compare the performance of the linear and nonlinear fitting methods. Simulation results show that the linear method can obtain only a suboptimal estimate of the unknown parameters and that the nonlinear method gives more accurate results. Application of the fitting methods is demonstrated where the water spectral attenuation coefficient is estimated from underwater images and imaging distances, which supports the improvement in the accuracy of parameter estimation by the nonlinear fitting method.
Adaptive Estimation Of Time-Varying Parameters In Linear Systems

yongliang zhu,prabhakar r pagilla

DOI: https://doi.org/10.1109/ACC.2003.1240489

2003-01-01

Abstract:Estimation of time-varying parameters via local polynomial approximations in linearly parameterized systems is considered. A strategy for approximating a time-varying parameter locally by a polynomial is given. The estimation time is divided into small intervals; in each interval the time-varying parameter is approximated by a time polynomial with unknown constant coefficients. A condition for resetting the parameter estimate at the beginning of each interval is derived; the condition ensures that the estimate of the time-varying parameter is continuous and allows for the coefficients of the polynomial to be different in various time intervals. Modifications of the least-squares and the gradient algorithms are provided to estimate the time-varying parameters. Stability of the proposed algorithms is shown and discussed. Simulation results on an example are given.
Adaptive Estimation of Time-Varying Parameters in Linearly Parametrized Systems

Yongliang Zhu,Prabhakar R. Pagilla

DOI: https://doi.org/10.1115/1.2234488

2006-01-01

Abstract:Adaptive estimation of time-varying parameters in linearly parametrized systems is considered. The estimation time is divided into small intervals; in each interval the time-varying parameter is approximated by a time polynomial with unknown coefficients. A condition for resetting of the parameter estimate at the beginning of each interval is derived; the condition guarantees that the estimate of the time-varying parameter is continuous and also allows for the coefficients of the polynomial to be different in various time intervals. A modified version of the least-squares algorithm is provided to estimate the time-varying parameters. Stability of the proposed algorithm is shown and discussed. Simulation results on an example are given to validate the proposed method.
Parameter estimation for nonlinear time-delay systems with noisy output measurements

Qun Lin,Ryan C. Loxton,Chao Xu,Kok Lay Teo

DOI: https://doi.org/10.1016/j.automatica.2015.06.028

IF: 6.4

2015-01-01

Automatica

Abstract:This paper considers the problem of using noisy output data to estimate unknown time-delays and unknown system parameters in a general nonlinear time-delay system. We formulate the problem as a dynamic optimization problem in which the unknown quantities are decision variables to be chosen optimally, with the cost function penalizing the mean and variance of the least-squares error between actual and predicted system output. Since the time-delays and system parameters influence the cost function implicitly through the governing time-delay system, the cost function's gradient-which is required to solve the problem using gradient-based optimization techniques-cannot be computed analytically using standard differentiation rules. We instead develop two computational methods for evaluating this gradient: one involves solving an auxiliary time-delay system forward in time; the other involves solving an auxiliary time-advance system backward in time. On this basis, we propose an efficient optimization algorithm for determining optimal estimates for the time-delays and system parameters. We conclude the paper by examining the performance of this algorithm on a dynamic model of a continuously-stirred tank reactor.
The Solution of High Accuracy Parameter Estimation Based on the Model of Enhanced Kalmen Filtering

CHU Ning,XING Shi-qi,ZHANG Kai

DOI: https://doi.org/10.3969/j.issn.1000-0984.2007.14.006

2007-01-01

Abstract:The paper analyzes the principal of the nonlinear least squares and propose a solution of high accuracy parameter estimation based on the model of enhanced Kalmen filtering.Applying the methods of the depth-first search algorithm and fast pruning,the calculation speed is improved.A two-objective programming model is made as the time-observation is inaccurate.
Parameter estimation for nonlinear dynamic systems from a feedback control viewpoint

Xuemei Zhu,Shuqing Wang,Pu Li,Wozny, G.

DOI: https://doi.org/10.1109/ICCA.2005.1528102

2005-01-01

Abstract:The estimation of time-variant parameters in nonlinear models remains as an open problem. In this work, we propose a new approach to addressing this problem by tuning weighting factors in the least-squares dynamic optimization. Through analysis of the solution procedure of the resulting optimization problem, it is found that parameter estimation with a weighted least-squares method is equivalent to a multi-variable closed-loop control system. The weighting factors in the objective function are equivalent to the controller parameters. Like any control system synthesis, the tuning can be made based on the output-input sensitivity analysis. The effectiveness of the approach is illustrated by estimating tray efficiencies and feed composition of a heat-integrated distillation column system.
Novel parameter identification algorithm for nonstationary stochastic system

HuaPing Dai,Jiawen Dong,Jixin Qian

1996-01-01

Tien Tzu Hsueh Pao/Acta Electronica Sinica

Abstract:A novel two stage least Square(TS-LS) algorithm has been proposed based on theoretical derivation for nonstationary dynamic systems with input slow time-varying disturbance, also, its performance has been theoretically analysed. Numerical simulation results show its efficiency and some important properties.
Non-linear Gradient Algorithm for Parameter Estimation: Extended version

Juan G. Rueda-Escobedo,Jaime A. Moreno

DOI: https://doi.org/10.1109/CDC.2015.7402855

2015-09-09

Abstract:Gradient algorithms are classical in adaptive control and parameter estimation. For instantaneous quadratic cost functions they lead to a linear time-varying dynamic system that converges exponentially under persistence of excitation conditions. In this paper we consider (instantaneous) non-quadratic cost functions, for which the gradient algorithm leads to non-linear (and non Lipschitz) time-varying dynamics, which are homogeneous in the state. We show that under persistence of excitation conditions they also converge globally, uniformly and asymptotically. Compared to the linear counterpart, they accelerate the convergence and can provide for finite-time or fixed-time stability.

Optimization and Control
Variable projection algorithms with sparse constraint for separable nonlinear models

Hui-Lang Xu,Guang-Yong Chen,Si-Qing Cheng,Min Gan,Jing Chen

DOI: https://doi.org/10.1007/s11768-023-00194-3

2024-01-26

Control Theory and Technology

Abstract:Separable nonlinear models are widely used in various fields such as time series analysis, system modeling, and machine learning, due to their flexible structures and ability to capture nonlinear behavior of data. However, identifying the parameters of these models is challenging, especially when sparse models with better interpretability are desired by practitioners. Previous theoretical and practical studies have shown that variable projection (VP) is an efficient method for identifying separable nonlinear models, but these are based on penalty of model parameters, which cannot be directly extended to deal with sparse constraint. Based on the exploration of the structural characteristics of separable models, this paper proposes gradient-based and trust-region-based variable projection algorithms, which mainly solve two key problems: how to eliminate linear parameters under sparse constraint; and how to deal with the coupling relationship between linear and nonlinear parameters in the model. Finally, numerical experiments on synthetic data and real time series data are conducted to verify the effectiveness of the proposed algorithms.

automation & control systems
Variable Projection Algorithms: Theoretical Insights and A Novel Approach for Problems with Large Residual

Guangyong Chen, Peng Xue, Min Gan, Jing Chen, Wenzhong Guo, C. L. Philip. Chen

2024-02-22

Abstract:This paper delves into an in-depth exploration of the Variable Projection (VP) algorithm, a powerful tool for solving separable nonlinear optimization problems across multiple domains, including system identification, image processing, and machine learning. We first establish a theoretical framework to examine the effect of the approximate treatment of the coupling relationship among parameters on the local convergence of the VP algorithm and theoretically prove that the Kaufman's VP algorithm can achieve a similar convergence rate as the Golub \& Pereyra's form. These studies fill the gap in the existing convergence theory analysis, and provide a solid foundation for understanding the mechanism of VP algorithm and broadening its application horizons. Furthermore, drawing inspiration from these theoretical revelations, we design a refined VP algorithm for handling separable nonlinear optimization problems characterized by large residual, called VPLR, which boosts the convergence performance by addressing the interdependence of parameters within the separable model and by continually correcting the approximated Hessian matrix to counteract the influence of large residual during the iterative process. The effectiveness of this refined algorithm is corroborated through numerical experimentation.

Optimization and Control
Linear Convergence Results for Inertial Type Projection Algorithm for Quasi-Variational Inequalities

Yonghong Yao,Lateef O. Jolaoso,Yekini Shehu

2024-04-22

Abstract:Many recently proposed gradient projection algorithms with inertial extrapolation step for solving quasi-variational inequalities in Hilbert spaces are proven to be strongly convergent with no linear rate given when the cost operator is strongly monotone and Lipschitz continuous. In this paper, our aim is to design an inertial type gradient projection algorithm for quasi-variational inequalities and obtain its linear rate of convergence. Therefore, our results fill in the gap for linear convergence results for inertial type gradient projection algorithms for quasi variational inequalities in Hilbert spaces. We perform numerical implementations of our proposed algorithm and give numerical comparisons with other related inertial type gradient projection algorithms for quasi variational inequalities in the literature.

Optimization and Control
Parameter estimation algorithms of linear systems with time-delays based on the frequency responses and harmonic balances under the multi-frequency sinusoidal signal excitation

Jiayao Ni,Yanliang Zhang,Feng Ding,Xi-Sheng Zhan,Tasawar Hayat

DOI: https://doi.org/10.1016/j.sigpro.2020.107904

IF: 4.729

2021-04-01

Signal Processing

Abstract:<p>In this paper, we study the parameter estimation problem of a class of linear time-delay systems. Based on the frequency responses and harmonic balance methods and by means of the gradient search, a two-stage stochastic gradient and gradient-based iterative algorithm is developed by using the collected data under the sinusoidal excitation. To improve the estimation accuracy, a two-stage multi-innovation stochastic gradient and gradient-based iterative algorithm is presented. Two simulation examples are provided to test the proposed algorithms.</p>

engineering, electrical & electronic
A Two-Step Spectral Gradient Projection Method for System of Nonlinear Monotone Equations and Image Deblurring Problems

Aliyu Muhammed Awwal,Lin Wang,Poom Kumam,Hassan Mohammad

DOI: https://doi.org/10.3390/sym12060874

2020-05-26

Symmetry

Abstract:In this paper, we propose a two-step iterative algorithm based on projection technique for solving system of monotone nonlinear equations with convex constraints. The proposed two-step algorithm uses two search directions which are defined using the well-known Barzilai and Borwein (BB) spectral parameters.The BB spectral parameters can be viewed as the approximations of Jacobians with scalar multiple of identity matrices. If the Jacobians are close to symmetric matrices with clustered eigenvalues then the BB parameters are expected to behave nicely. We present a new line search technique for generating the separating hyperplane projection step of Solodov and Svaiter (1998) that generalizes the one used in most of the existing literature. We establish the convergence result of the algorithm under some suitable assumptions. Preliminary numerical experiments demonstrate the efficiency and computational advantage of the algorithm over some existing algorithms designed for solving similar problems. Finally, we apply the proposed algorithm to solve image deblurring problem.

multidisciplinary sciences
On line parameter estimation based on gradient algorithm for fractional order systems.

Yiheng Wei,Zhenyuan Sun,Yangsheng Hu,Yong Wang

DOI: https://doi.org/10.1080/23307706.2015.1050466

2015-01-01

Abstract:This article investigates the problem of on line parameter estimation for fractional-order linear systems. Based on the theory of fractional-order calculus, the conventional gradient estimator is extended to the fractional-order areas. The stability of the estimator and the convergence are analysed using the continuous frequency distributed model and indirect Lyapunov method. Finally, numerical simulation examples are given to demonstrate the effectiveness of the proposed schemes.
An efficient and robust varying-parameter projection neural network for sparse signal reconstruction

Qing Hu,Bing Zheng

DOI: https://doi.org/10.1016/j.neucom.2024.127939

IF: 6

2024-05-30

Neurocomputing

Abstract:Sparse signal reconstruction plays an important role in many practical applications, and therefore, many scholars strive to study its various effective (approximate) solution approaches, including some projection neural network models with finite-time or fixed-time convergence due to their parallel nature and convenience for hardware implementation. However, we experimentally find that noise has a significant influence on the convergence performance of these models. To overcome this problem, in this paper, we propose a new efficient varying-parameter projection neural network model (VPPNN) using the sliding mode control technique and a novel time-varying parameter for sparse signal reconstruction. Theoretical analysis shows that this new model not only has shorter fixed convergence time under certain conditions but also has noise-tolerance when compared to the existing projection neural network models. Numerical simulation experiments (via sparse signal/image reconstruction) are performed to illustrate the superiority of the new proposed model over the existing ones in terms of the convergence rate and robustness.

computer science, artificial intelligence
Hierarchical Stochastic Gradient Algorithm and its Performance Analysis for a Class of Bilinear-in-Parameter Systems

Feng Ding,Xuehai Wang

DOI: https://doi.org/10.1007/s00034-016-0367-7

2016-07-21

Abstract:This paper considers the parameter identification for a special class of nonlinear systems, i.e., bilinear-in-parameter systems. Based on the hierarchical identification principle, a hierarchical stochastic gradient (HSG) estimation algorithm is presented. The basic idea is to decompose a bilinear-in-parameter system into two subsystems and to derive the HSG identification algorithm for estimating the system parameters by replacing the unknown variables in the information vectors with their estimates obtained at the previous time. The convergence analysis of the proposed algorithm indicates that the parameter estimation errors converge to zero under persistent excitation conditions. The simulation results show that the proposed algorithm is effective.

engineering, electrical & electronic
Efficient Parameters Estimation Method for the Separable Nonlinear Least Squares Problem

Ke Wang,Guolin Liu,Qiuxiang Tao,Min Zhai

DOI: https://doi.org/10.1155/2020/9619427

IF: 2.3

2020-02-10

Complexity

Abstract:In this work, we combine the special structure of the separable nonlinear least squares problem with a variable projection algorithm based on singular value decomposition to separate linear and nonlinear parameters. Then, we propose finding the nonlinear parameters using the Levenberg–Marquart (LM) algorithm and either solve the linear parameters using the least squares method directly or by using an iteration method that corrects the characteristic values based on the L-curve, according to whether or not the nonlinear function coefficient matrix is ill posed. To prove the feasibility of the proposed method, we compared its performance on three examples with that of the LM method without parameter separation. The results show that (1) the parameter separation method reduces the number of iterations and improves computational efficiency by reducing the parameter dimensions and (2) when the coefficient matrix of the linear parameters is well-posed, using the least squares method to solve the fitting problem provides the highest fitting accuracy. When the coefficient matrix is ill posed, the method of correcting characteristic values based on the L-curve provides the most accurate solution to the fitting problem.

mathematics, interdisciplinary applications,multidisciplinary sciences
Monotonous Parameter Estimation of One Class of Nonlinearly Parameterized Regressions without Overparameterization

Anton Glushchenko,Konstantin Lastochkin

2023-08-21

Abstract:The estimation law of unknown parameters vector ${\theta}$ is proposed for one class of nonlinearly parametrized regression equations $y\left( t \right) = \Omega \left( t \right)\Theta \left( \theta \right)$. We restrict our attention to parametrizations that are widely obtained in practical scenarios when polynomials in $\theta$ are used to form $\Theta \left( \theta \right)$. For them we introduce a new 'linearizability' assumption that a mapping from overparametrized vector of parameters $\Theta \left( \theta \right)$ to original one $\theta$ exists in terms of standard algebraic functions. Under such assumption and weak requirement of the regressor finite excitation, on the basis of dynamic regressor extension and mixing technique we propose a procedure to reduce the nonlinear regression equation to the linear parameterization without application of singularity causing operations and the need to identify the overparametrized parameters vector. As a result, an estimation law with exponential convergence rate is derived, which, unlike known solutions, (i) does not require a strict P-monotonicity condition to be met and a priori information about $\theta$ to be known, (ii) ensures elementwise monotonicity for the parameter error vector. The effectiveness of our approach is illustrated with both academic example and 2-DOF robot manipulator control problem.

Systems and Control
A three-term subspace projection method for solving systems of nonlinear monotone equations

Yong Zhao,Mengjiao Niu,Jinkui Liu

DOI: https://doi.org/10.3934/jimo.2024156

2024-10-31

Journal of Industrial and Management Optimization

Abstract:In this paper, a three-term derivative-free projection algorithm for solving nonlinear monotone equations is proposed. This algorithm is established by minimizing a quadratic approximation of the objective function in a subspace and combining with the hyperplane projection technique. Under some proper conditions, the global convergence of the algorithm is proved. Numerical results show that the performance of this algorithm is better than some existing algorithms.

engineering, multidisciplinary,operations research & management science,mathematics, interdisciplinary applications

The projection algorithm for estimation of nonlinear system parameter

Identification of nonlinearly parameterized nonlinear models: application to mass balance systems.

Parameter Estimation in Exponential Models by Linear and Nonlinear Fitting Methods.

Adaptive Estimation Of Time-Varying Parameters In Linear Systems

Adaptive Estimation of Time-Varying Parameters in Linearly Parametrized Systems

Parameter estimation for nonlinear time-delay systems with noisy output measurements

The Solution of High Accuracy Parameter Estimation Based on the Model of Enhanced Kalmen Filtering

Parameter estimation for nonlinear dynamic systems from a feedback control viewpoint

Novel parameter identification algorithm for nonstationary stochastic system

Non-linear Gradient Algorithm for Parameter Estimation: Extended version

Variable projection algorithms with sparse constraint for separable nonlinear models

Variable Projection Algorithms: Theoretical Insights and A Novel Approach for Problems with Large Residual

Linear Convergence Results for Inertial Type Projection Algorithm for Quasi-Variational Inequalities

Parameter estimation algorithms of linear systems with time-delays based on the frequency responses and harmonic balances under the multi-frequency sinusoidal signal excitation

A Two-Step Spectral Gradient Projection Method for System of Nonlinear Monotone Equations and Image Deblurring Problems

On line parameter estimation based on gradient algorithm for fractional order systems.

An efficient and robust varying-parameter projection neural network for sparse signal reconstruction

Hierarchical Stochastic Gradient Algorithm and its Performance Analysis for a Class of Bilinear-in-Parameter Systems

Efficient Parameters Estimation Method for the Separable Nonlinear Least Squares Problem

Monotonous Parameter Estimation of One Class of Nonlinearly Parameterized Regressions without Overparameterization

A three-term subspace projection method for solving systems of nonlinear monotone equations