Abstract:Iterative sketching and sketch-and-precondition are well-established randomized algorithms for solving large-scale, over-determined linear least-squares problems. In this paper, we introduce a new perspective that interprets Iterative Sketching and Sketching-and-Precondition as forms of Iterative Refinement. We also examine the numerical stability of two distinct refinement strategies, iterative refinement and recursive refinement, which progressively improve the accuracy of a sketched linear solver. Building on this insight, we propose a novel algorithm, Sketched Iterative and Recursive Refinement (SIRR), which combines both refinement methods. SIRR demonstrates a \emph{four order of magnitude improvement} in backward error compared to iterative sketching, achieved simply by reorganizing the computational order, ensuring that the computed solution exactly solves a modified least-squares system where the coefficient matrix deviates only slightly from the original matrix. To the best of our knowledge, \emph{SIRR is the first asymptotically fast, single-stage randomized least-squares solver that achieves both forward and backward stability}.

Wilkinson’s Iterative Refinement of Solution with Automatic Step-Size Control for Linear System of Equations

A Note on Wilkinson's Iterative Refinement of Solution with Automatic Step-Size Control

Extended version with the analysis of dynamic system for iterative refinement of solution

New Iterative Improvement of a Solution for an Ill-conditioned System of Linear Equations Based on a Linear Dynamic System

Iterative Refinement of Solution with Biparameter for Solving Ill-Conditioned Systems of Linear Algebraic Equations

A New Iterative Refinement with Roundoff Error Analysis.

Stable iterative refinement algorithms for solving linear systems

A NUMERICAL METHOD OF STIFF PROBLEMS FOR ILL-CONDITIONED LINEAR SYSTEMS OF ALGEBRAIC EQUATIONS

Note on Predictor-Corrector Process for Ill-Conditioned Linear System of Equations

Note on the Improvement of Newton’s Method for System of Nonlinear Equations

A Smoothing-Type Algorithm for Solving System of Inequalities

Local and Global Behavior for Algorithms of Solving Equations

Control-theoretic Design of Iterative Methods for Symmetric Linear Systems of Equations

A Control-Theoretic Study on Iterative Solutions to Nonlinear Equations for Applications in Embedded Systems.

An improved iterative algorithm for solving optimal tracking control problems of stochastic systems

A Significant Improvement On Newton'S Iterative Method

Randomized Iterative Solver as Iterative Refinement: A Simple Fix Towards Backward Stability

Error analysis for the predictor–corrector process relating to ill-conditioned linear system of equations

Adaptive Iterative Learning Control for Discrete-Time Systems with Non-Repetitiveness

A Successive over Relaxation Implicit Iterative Algorithm for Solving Stochastic Linear Systems with Markov Jumps

Developing iterative algorithms to solve Sylvester tensor equations