Abstract:In this paper we show how to accelerate randomized coordinate descent methods and achieve faster convergence rates without paying per-iteration costs in asymptotic running time. In particular, we show how to generalize and efficiently implement a method proposed by Nesterov, giving faster asymptotic running times for various algorithms that use standard coordinate descent as a black box. In addition to providing a proof of convergence for this new general method, we show that it is numerically stable, efficiently implementable, and in certain regimes, asymptotically optimal. To highlight the power of this algorithm, we show how it can used to create faster linear system solvers in several regimes: • We show how this method achieves a faster asymptotic runtime than conjugate gradient for solving a broad class of symmetric positive definite systems of equations. • We improve the convergence guarantees for Kaczmarz methods, a popular technique for image reconstruction and solving overdetermined systems of equations, by accelerating an algorithm of Strohmer and Vershynin. • We achieve the best known running time for solving Symmetric Diagonally Dominant (SDD) system of equations in the unit-cost RAM model, obtaining a running time of ${\bf O}({\bf m}\log^{\wedge}\{3/2\}{\bf n}(\log\log {\bf n})^{\wedge}\{1/2\}\log((\log {\bf n})/{\bf eps}))$ by accelerating a recent solver by Kelner et al. We show how this method achieves a faster asymptotic runtime than conjugate gradient for solving a broad class of symmetric positive definite systems of equations. We improve the convergence guarantees for Kaczmarz methods, a popular technique for image reconstruction and solving overdetermined systems of equations, by accelerating an algorithm of Strohmer and Vershynin. We achieve the best known running time for solving Symmetric Diagonally Dominant (SDD) system of equations in the unit-cost RAM model, obtaining a running time of ${\bf O}({\bf m}\log^{\wedge}\{3/2\}{\bf n}(\log\log {\bf n})^{\wedge}\{1/2\}\log((\log {\bf n})/{\bf eps}))$ by accelerating a recent solver by Kelner et al. Beyond the independent interest of these solvers, we believe they highlight the versatility of the approach of this paper and we hope that they will open the door for further algorithmic improvements in the future.

Poster: Star (Space-Time Adaptive And Reductive) Algorithms For Real-World Space-Time Optimality

Brief Announcement: Star (Space-Time Adaptive And Reductive) Algorithms For Dynamic Programming Recurrences With More Than O(1) Dependency

Time-sharing Parallel Applications Through Performance-Targeted Feedback-Controlled Real-Time Scheduling.

Improving the Space-Time Efficiency of Matrix Multiplication Algorithms

Programmatic Control of a Compiler for Generating High-performance Spatial Hardware

A Parallel Dynamic Programming Algorithm on a Multi-Core Architecture

Tight Bounds for Low Dimensional Star Stencils in the Parallel External Memory Model

Efficient Accelerated Coordinate Descent Methods and Faster Algorithms for Solving Linear Systems

Research on Dynamic Optimization Method of Embedded Multi-core Performance based on Runtime

Improving the Space-Time Efficiency of Processor-Oblivious Matrix Multiplication Algorithms

Competitive parallelism: getting your priorities right

Tight Time Complexities in Parallel Stochastic Optimization with Arbitrary Computation Dynamics

Optimal Time Complexities of Parallel Stochastic Optimization Methods Under a Fixed Computation Model

Cache-oblivious Wavefront: Improving Parallelism of Recursive Dynamic Programming Algorithms Without Losing Cache-Efficiency

Competitive Implementation of Parallel Programs

ON PARALLEL PROGRAMMING AND OPTIMISATION FOR MULTI-CORE

Optimizing Stencil Code Via Locality Of Computation

An Asynchronous Algorithm for Optimizing the Communication Performance

A Run-Time Optimization Approach for Reducing Data Movements Using Locality-Aware Searching

Dynamic Stencil: Effective Exploitation of Run-Time Resources in Reconfigurable Clusters

A Study of Performance Programming of CPU, GPU accelerated Computers and SIMD Architecture