Abstract:In this paper we show how to accelerate randomized coordinate descent methods and achieve faster convergence rates without paying per-iteration costs in asymptotic running time. In particular, we show how to generalize and efficiently implement a method proposed by Nesterov, giving faster asymptotic running times for various algorithms that use standard coordinate descent as a black box. In addition to providing a proof of convergence for this new general method, we show that it is numerically stable, efficiently implementable, and in certain regimes, asymptotically optimal. To highlight the power of this algorithm, we show how it can used to create faster linear system solvers in several regimes: • We show how this method achieves a faster asymptotic runtime than conjugate gradient for solving a broad class of symmetric positive definite systems of equations. • We improve the convergence guarantees for Kaczmarz methods, a popular technique for image reconstruction and solving overdetermined systems of equations, by accelerating an algorithm of Strohmer and Vershynin. • We achieve the best known running time for solving Symmetric Diagonally Dominant (SDD) system of equations in the unit-cost RAM model, obtaining a running time of ${\bf O}({\bf m}\log^{\wedge}\{3/2\}{\bf n}(\log\log {\bf n})^{\wedge}\{1/2\}\log((\log {\bf n})/{\bf eps}))$ by accelerating a recent solver by Kelner et al. We show how this method achieves a faster asymptotic runtime than conjugate gradient for solving a broad class of symmetric positive definite systems of equations. We improve the convergence guarantees for Kaczmarz methods, a popular technique for image reconstruction and solving overdetermined systems of equations, by accelerating an algorithm of Strohmer and Vershynin. We achieve the best known running time for solving Symmetric Diagonally Dominant (SDD) system of equations in the unit-cost RAM model, obtaining a running time of ${\bf O}({\bf m}\log^{\wedge}\{3/2\}{\bf n}(\log\log {\bf n})^{\wedge}\{1/2\}\log((\log {\bf n})/{\bf eps}))$ by accelerating a recent solver by Kelner et al. Beyond the independent interest of these solvers, we believe they highlight the versatility of the approach of this paper and we hope that they will open the door for further algorithmic improvements in the future.

Acceleration analysis of the quadratic sieve method based on the online matrix solving

Acceleration of Euclidean Algorithm and Extensions

An efficient method for integer factorization

Smooth Subsum Search: A heuristic for practical integer factorization

Acceleration of Euclidean Algorithm and Rational Number Reconstruction

On Two Iteration Methods for the Quadratic Matrix Equations

Improved Non-Sieving Quadratic Sieve

Quadratic Sieve Factorization Quantum Algorithm and its Simulation

A decomposition algorithm for solving large-scale quadratic programming problems

Matrix Multiplication with Less Arithmetic Complexity and IO Complexity

Performance evaluation of accelerated complex multiple-precision LU decomposition

OPTIMIZATION OF THE QUADRATIC SIEVE IN LARGE INTEGER FACTORIZATION

An Empirical Comparison of the Quadratic Sieve Factoring Algorithm and the Pollard Rho Factoring Algorithm

Two Efficient Lopsided Double-Step Methods for Solving Complex Symmetric Linear Systems

An accelerated active-set algorithm for a quadratic semidefinite program with general constraints

A structure-preserving doubling algorithm for solving a class of quadratic matrix equation with $M$-matrix

Improving the numerical stability of fast matrix multiplication

Efficient Accelerated Coordinate Descent Methods and Faster Algorithms for Solving Linear Systems

Coupled Iterative Algorithms Based on Optimisation for Solving Sylvester Matrix Equations

An Accelerated Block Randomized Kaczmarz Method

Iterative methods for solving factorized linear systems