Hybrid Quantum-Classical Scheduling for Accelerating Neural Network Training with Newton's Gradient Descent

Pingzhi Li,Junyu Liu,Hanrui Wang,Tianlong Chen
2024-05-01
Abstract:Optimization techniques in deep learning are predominantly led by first-order gradient methodologies, such as SGD. However, neural network training can greatly benefit from the rapid convergence characteristics of second-order optimization. Newton's GD stands out in this category, by rescaling the gradient using the inverse Hessian. Nevertheless, one of its major bottlenecks is matrix inversion, which is notably time-consuming in $O(N^3)$ time with weak scalability.
Quantum Physics,Artificial Intelligence,Machine Learning
What problem does this paper attempt to address?