Thermodynamic Algorithms for Quadratic Programming

Patryk-Lipka Bartosik,Kaelan Donatella,Maxwell Aifer,Denis Melanson,Marti Perarnau-Llobet,Nicolas Brunner,Patrick J. Coles

2024-11-21

Abstract:Thermodynamic computing has emerged as a promising paradigm for accelerating computation by harnessing the thermalization properties of physical systems. This work introduces a novel approach to solving quadratic programming problems using thermodynamic hardware. By incorporating a thermodynamic subroutine for solving linear systems into the interior-point method, we present a hybrid digital-analog algorithm that outperforms traditional digital algorithms in terms of speed. Notably, we achieve a polynomial asymptotic speedup compared to conventional digital approaches. Additionally, we simulate the algorithm for a support vector machine and predict substantial practical speedups with only minimal degradation in solution quality. Finally, we detail how our method can be applied to portfolio optimization and the simulation of nonlinear resistive networks.

Emerging Technologies,Statistical Mechanics,Optimization and Control

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is to use thermodynamic computing hardware to accelerate the solution of quadratic programming (QP) problems. Specifically, the paper introduces a novel method. By using thermodynamic subroutines to solve linear systems and combining with the interior - point method, a hybrid digital - analog algorithm is proposed to achieve a faster solution speed than traditional digital algorithms. The following are the key points of this research:

### Research Background and Motivation

1. **Universality of Optimization Problems**: Hard optimization problems are widespread in fields such as machine learning, artificial intelligence, finance, engineering, and bioinformatics.
2. **Limitations of Digital Computers**:
- Digital devices are discrete in nature and it is difficult to efficiently explore the solution space of continuous optimization problems.
- Digital operations require active energy injection, resulting in high energy consumption.

### Advantages of Thermodynamic Computing

1. **Natural Optimization Characteristics**: Thermodynamic computers optimize naturally through the inherent processes of physical systems and can effectively solve certain optimization problems.
2. **Existing Applications**: In recent years, thermodynamic computing has been applied in fields such as neural network modeling, generative AI, and Bayesian inference.

### Contributions of This Paper

1. **Application of Thermodynamic Algorithms**: The paper shows how to apply thermodynamic computing hardware to quadratic programming problems, especially by offloading the linear system - solving subroutines to the stochastic processing unit (SPU) while using the GPU for matrix - vector and matrix - matrix multiplications.
2. **Performance Improvement**: Based on realistic hardware assumptions, it is predicted that for support vector machine tasks with approximately 1,000 dimensions, this method can achieve a 10 - 30 - fold speed improvement with only a slight decrease in solution quality.
3. **Application Scenarios**:
- **Support Vector Machine (SVM)**: Classification tasks in machine learning.
- **Portfolio Optimization**: Optimizing asset allocation in the financial field.
- **Non - linear Resistor Network Simulation**: Simulating AI applications in hardware design.

### Technical Details

1. **Unconstrained and Equality - Constrained QP**:
- For the unconstrained case, the optimal solution \( x^\star = Q^{-1}c \) can be solved analytically.
- For the equality - constrained case, it is transformed into an unconstrained problem by introducing the Lagrange multiplier method and the corresponding linear equations are solved.
2. **Inequality - Constrained QP**:
- It is solved using the interior - point method, where the most time - consuming step is to calculate the direction vector \( \Delta r \) of the Newton step.
- The overall efficiency is improved by accelerating the linear system solution with the thermodynamic linear solver (TLS).

### Computational Complexity

- **LU Decomposition**: \( O((n + m)^{7/2}\log(1/\epsilon)) \)
- **Conjugate Gradient Method**: \( O(n^3 + m^3+\sqrt{n + m}\log(1/\epsilon)[n^2 + m^2+\sqrt{\kappa}(n + m)^2]) \)
- **Thermodynamic Algorithm**: \( O(n^3 + m^3+\sqrt{n + m}\log(1/\epsilon)[n^2 + m^2+\kappa^2(n + m)\delta^{-2}]) \)

### Experimental Results

- Training an SVM model on the UCI breast cancer dataset, the results show that the thermodynamic algorithm is significantly superior to traditional digital methods, especially achieving a 10 - 30 - fold speed improvement on medium - sized datasets.

In conclusion, this paper provides a fast and energy - efficient solution to quadratic programming problems by introducing thermodynamic computing hardware and shows its potential in multiple practical applications.