Abstract:Interior point methods are widely used for different types of mathematical optimization problems. Many implementations of interior point methods in use today rely on direct linear solvers to solve systems of equations in each iteration. The need to solve ever larger optimization problems more efficiently and the rise of hardware accelerators for general purpose computing has led to a large interest in using iterative linear solvers instead, with the major issue being inevitable ill-conditioning of the linear systems arising as the optimization progresses. We investigate the use of Krylov solvers for interior point methods in solving optimization problems from radiation therapy and support vector machines. We implement a prototype interior point method using a so called doubly augmented formulation of the Karush-Kuhn-Tucker linear system of equations, originally proposed by Forsgren and Gill, and evaluate its performance on real optimization problems from radiation therapy and support vector machines. Crucially, our implementation uses a preconditioned conjugate gradient method with Jacobi preconditioning internally. Our measurements of the conditioning of the linear systems indicate that the Jacobi preconditioner improves the conditioning of the systems to a degree that they can be solved iteratively, but there is room for further improvement in that regard. Furthermore, profiling of our prototype code shows that it is suitable for GPU acceleration, which may further improve its performance in practice. Overall, our results indicate that our method can find solutions of acceptable accuracy in reasonable time, even with a simple Jacobi preconditioner.

A comparison of Krylov methods for Shifted Skew-Symmetric Systems

On Krylov subspace methods for skew-symmetric and shifted skew-symmetric linear systems

Iterative Solution of Multi-Shifted Linear Systems with Multiple Right-Hand Sides.

Flexible and deflated variants of the block shifted GMRES method

Sketched and Truncated Polynomial Krylov Subspace Methods: Matrix Sylvester Equations

Randomized Complete Pivoting for Solving Symmetric Indefinite Linear Systems

Solving Shifted Linear Systems with Restarted GMRES Augmented with Error Approximations.

Sketched and truncated polynomial Krylov methods: Evaluation of matrix functions

Krylov Solvers for Interior Point Methods with Applications in Radiation Therapy and Support Vector Machines

On restarted and deflated block FOM and GMRES methods for sequences of shifted linear systems

Obtaining the pseudoinverse solution of singular range-symmetric linear systems with GMRES-type methods

Iterative Methods for Sparse Symmetric Multilinear Systems

A flexible and adaptive Simpler GMRES with deflated restarting for shifted linear systems

Efficient Variants of the CMRH Method for Solving a Sequence of Multi-Shifted Non-Hermitian Linear Systems Simultaneously

Acceleration of the Scheduled Relaxation Jacobi Method: Promising Strategies for Solving Large, Sparse Linear Systems

Global GMRES with Deflated Restarting for Families of Shifted Linear Systems

Memory-efficient recycling of large Krylov-subspaces for sequences of Hermitian linear systems

Convergence Properties Of Krylov Subspace Methods For Singular Linear Systems With Arbitrary Index

A Non-Krylov subspace Method for Solving Large and Sparse Linear System of Equations

Practical shift choice in the shift-and-invert Krylov subspace evaluations of the matrix exponential

Novel minimum residual MHSS iteration method for solving complex symmetric linear systems