Abstract:We provide a complete characterization of the randomized Kaczmarz algorithm (RKA) for inconsistent linear systems. The Kaczmarz algorithm, known in some fields as the algebraic reconstruction technique, is a classical method for solving large-scale overdetermined linear systems through a sequence of projection operators; the randomized Kaczmarz algorithm is a recent proposal by Strohmer and Vershynin to randomize the sequence of projections in order to guarantee exponential convergence (in mean square) to the solutions. A flurry of work followed this development, with renewed interest in the algorithm, its extensions, and various bounds on their performance. Earlier, we studied the special case of consistent linear systems and provided an exact formula for the mean squared error (MSE) in the value reconstructed by RKA, as well as a simple way to compute the exact decay rate of the error. In this work, we consider the case of inconsistent linear systems, which is a more relevant scenario for most applications. First, by using a "lifting trick", we derive an exact formula for the MSE given a fixed noise vector added to the measurements. Then we show how to average over the noise when it is drawn from a distribution with known first and second-order statistics. Finally, we demonstrate the accuracy of our exact MSE formulas through numerical simulations, which also illustrate that previous upper bounds in the literature may be several orders of magnitude too high.

Windowing and random weighting based cubature RTS smoothing for target tracking

Bearing-Only Target Tracking Using Cubature Rauch-Tung-Striebel Smoother

Novel Roughening Algorithm and Hardware Architecture for Bearings-Only Tracking Using Particle Filter

Square-root RTS smoothing algorithms

Adaptive Neural Dynamic Surface Control With Prespecified Tracking Accuracy of Uncertain Stochastic Nonstrict-Feedback Systems

New square-root smoothing algorithms

Robust Filtering with Randomly Delayed Measurements and Its Application to Ballistic Target Tracking in Boost Phase.

RTS Smoother for GLMB filter

Backward smoothing for precise GNSS applications

Strong Tracking Adaptive Square-Root Cubatu re Kal man Filter Algorith m

Robust Visual Cooperative Tracking Using Constrained Adaptive Sparse Representations and Sparse Classifier Grids

Minimum Error Entropy Rauch-Tung-Striebel Smoother

Target tracking algorithm for system with multiplicative noises and randomly delayed measurements

A Novel Tracking Algorithm Based on Waveform Selection for Maneuvering Targets in Clutter

A new state estimation approach-Adaptive Fading Cubature Kalman filter

Robust Adaptive Radar Beamforming Based on Iterative Training Sample Selection Using Kurtosis of Generalized Inner Product Statistics

An Adaptive Feature Channel Weighting Scheme For Correlation Tracking

Treatment of Statistical Estimation Problems in Randomized Smoothing for Adversarial Robustness

Randomized Kaczmarz Algorithm for Inconsistent Linear Systems: An Exact MSE Analysis

An Adaptive Cubature Kalman Filter Based on Resampling-Free Sigma-Point Update Framework and Improved Empirical Mode Decomposition for INS/CNS Navigation

RTSNet: Learning to Smooth in Partially Known State-Space Models (Preprint)