Abstract:De facto, signal processing is the interpolation and extrapolation of a sequence of observations viewed as a realization of a stochastic process. Its role in applied statistics ranges from scenarios in forecasting and time series analysis to image reconstruction, machine learning, and the degradation modeling for reliability assessment. This topic, which has an old and honorable history dating back to the times of Gauss and Legendre, should therefore be of interest to readers of Technometrics. A general solution to the problem of filtering and prediction entails some formidable mathematics. Efforts to circumvent the mathematics has resulted in the need for introducing more explicit descriptions of the underlying process. One such example, and a noteworthy one, is the Kalman filter model, which is a special case of state space models or what statisticians refer to as dynamic linear models. Implementing the Kalman filter model in the era of “big and high velocity non-Gaussian data” can pose computational challenges with respect to efficiency and timeliness. Particle filtering is a way to ease such computational burdens. The purpose of this article is to trace the historical evolution of this development from its inception to its current state, with an expository focus on two versions of the particle filter, namely, the propagate first-update next and the update first-propagate next version. By way of going beyond a pure review, this article also makes transparent the importance and the role of a less recognized principle, namely, the principle of conditionalization, in filtering and prediction based on Bayesian methods. Furthermore, the article also articulates the philosophical underpinnings of the filtering and prediction set-up, a matter that needs to be made explicit, and Yule's decomposition of a random variable in terms of a sequence of innovations.

A general approach of least squares estimation and optimal filtering

Input and state estimation for linear systems: A least squares estimation approach

Synthesis of quasi-optimal fast filters by the least square criterion

A Tutorial on Linear Least Square Estimation

From Least Squares to Signal Processing and Particle Filtering

Optimal Filtering of Continuous-Time Stationary Processes by Means of the Backward Innovation Process

Linear Least-Squares Estimation of Discrete-Time Stationary Processes by Means of Backward Innovations

Optimal Linear Filtering for Discrete-Time Systems with Infinite-Dimensional Measurements

Optimal sampling for least squares approximation with general dictionaries

Kernel Based Filter for Nonlinear Estimation.

Least Squares Approximations in Linear Statistical Inverse Learning Problems

A new method for the nonlinear transformation of means and covariances in filters and estimators

Regularized Least-Mean-Square Algorithms

Optimal nonlinear filtering using the finite-volume method

Some Reduced-Order Non-Riccati Equations for Linear Least-Squares Estimation : the Stationary, Single-Output Case†

Least squares formulation for ill-posed inverse problems and applications

An Elementary Introduction to Kalman Filtering

A new approach to single-tone frequency estimation via linear least squares curve fitting

A Spectral Decomposition Method for the Linear Least-Squares Adaptive Filters

Variations on least squares

On the Generalization of Linear Least Mean Squares Estimation to Quantum Systems with Non-Commutative Outputs