Abstract:In this paper, we evaluate the different fully homomorphic encryption schemes, propose an implementation, and numerically analyze the applicability of gradient descent algorithms to solve quadratic programming in a homomorphic encryption setup. The limit on the multiplication depth of homomorphic encryption circuits is a major challenge for iterative procedures such as gradient descent algorithms. Our analysis not only quantifies these limitations on prototype examples, thus serving as a benchmark for future investigations, but also highlights additional trade-offs like the ones pertaining the choice of gradient descent or accelerated gradient descent methods, opening the road for the use of homomorphic encryption techniques in iterative procedures widely used in optimization based control. In addition, we argue that, among the available homomorphic encryption schemes, the one adopted in this work, namely CKKS, is the only suitable scheme for implementing gradient descent algorithms. The choice of the appropriate step size is crucial to the convergence of the procedure. The paper shows firsthand the feasibility of homomorphically encrypted gradient descent algorithms.

What problem does this paper attempt to address?

The paper attempts to address the problem of implementing the gradient descent algorithm to solve Quadratic Programming (QP) problems in a Homomorphic Encryption (HE) environment. Specifically, the paper focuses on the following points: 1. **Evaluating different fully homomorphic encryption schemes**: The paper first evaluates several existing fully homomorphic encryption schemes (such as BFV, BGV, CKKS, etc.) and proposes an implementation method. 2. **Numerical analysis of the applicability of the gradient descent algorithm**: In a homomorphic encryption environment, the applicability of the gradient descent algorithm is affected by the multiplication depth limitation. The paper quantifies these limitations through numerical analysis and provides benchmarks for future research. 3. **Choosing the appropriate gradient descent method**: The paper discusses the choice between ordinary Gradient Descent (GD) and Accelerated Gradient Descent (AGD) methods in a homomorphic encryption environment. Specifically, the paper points out that the condition number of the matrix is crucial in determining which method to choose. 4. **Implementing an efficient matrix multiplication algorithm**: To overcome the multiplication depth limitation in homomorphic encryption, the paper proposes a new matrix multiplication algorithm that is more efficient in terms of multiplication depth compared to existing methods. 5. **Validating the effectiveness of the algorithm**: Through numerical experiments, the paper validates the performance of the proposed algorithm under different condition numbers and demonstrates that the accelerated gradient descent method performs better under high condition numbers, while the ordinary gradient descent method performs better under low condition numbers due to the ability to perform more iterations. Overall, the paper aims to explore and address the technical challenges of implementing the gradient descent algorithm to solve Quadratic Programming problems in a homomorphic encryption environment, particularly the issues arising from multiplication depth limitations.

Homomorphically encrypted gradient descent algorithms for quadratic programming

Privacy-Preserving Logistic Regression Training with A Faster Gradient Variant

Functional Encryption with Application to Machine Learning: Simple Conversions from Generic Functions to Quadratic Functions

Convex quartic problems: homogenized gradient method and preconditioning

Toward Lossless Homomorphic Encryption for Scientific Computation

Quadratic Gradient: Combining Gradient Algorithms and Newton's Method as One

Privacy-Preserving Logistic Regression Training on Large Datasets

A dual gradient-projection method for large-scale strictly convex quadratic problems

Secure numerical simulations using fully homomorphic encryption

Fully homomorphic encryption scheme and Fermat's little theorem

On a Family of Relaxed Gradient Descent Methods for Quadratic Minimization

Extended convexity and smoothness and their applications in deep learning

Logistic regression model training based on the approximate homomorphic encryption

Analysis of Gradient Descent with Varying Step Sizes using Integral Quadratic Constraints

Quantum spectral method for gradient and Hessian estimation

A Generic Optimization Method of Multivariate Systems on Graphic Processing Units

A (Leveled) fully homomorphic encryption scheme based on error-free approximate GCD

Global Convergence of Non-Convex Gradient Descent for Computing Matrix Squareroot

First Order Methods beyond Convexity and Lipschitz Gradient Continuity with Applications to Quadratic Inverse Problems

Decentralized Stochastic Optimization with Inherent Privacy Protection