Abstract:In this paper, we present a practical solution to implement privacy-preserving CNN training based on mere Homomorphic Encryption (HE) technique. To our best knowledge, this is the first attempt successfully to crack this nut and no work ever before has achieved this goal. Several techniques combine to accomplish the task:: (1) with transfer learning, privacy-preserving CNN training can be reduced to homomorphic neural network training, or even multiclass logistic regression (MLR) training; (2) via a faster gradient variant called $\texttt{Quadratic Gradient}$, an enhanced gradient method for MLR with a state-of-the-art performance in convergence speed is applied in this work to achieve high performance; (3) we employ the thought of transformation in mathematics to transform approximating Softmax function in the encryption domain to the approximation of the Sigmoid function. A new type of loss function termed $\texttt{Squared Likelihood Error}$ has been developed alongside to align with this change.; and (4) we use a simple but flexible matrix-encoding method named $\texttt{Volley Revolver}$ to manage the data flow in the ciphertexts, which is the key factor to complete the whole homomorphic CNN training. The complete, runnable C++ code to implement our work can be found at: \href{<a class="link-external link-https" href="https://github.com/petitioner/HE.CNNtraining" rel="external noopener nofollow">this https URL</a>}{$\texttt{<a class="link-external link-https" href="https://github.com/petitioner/HE.CNNtraining" rel="external noopener nofollow">this https URL</a>}$}. We select $\texttt{REGNET\_X\_400MF}$ as our pre-trained model for transfer learning. We use the first 128 MNIST training images as training data and the whole MNIST testing dataset as the testing data. The client only needs to upload 6 ciphertexts to the cloud and it takes $\sim 21$ mins to perform 2 iterations on a cloud with 64 vCPUs, resulting in a precision of $21.49\%$.

Privacy-Preserving Logistic Regression Training on Large Datasets

Privacy-Preserving Logistic Regression Training on Large Datasets

Privacy-Preserving Logistic Regression Training with A Faster Gradient Variant

When Homomorphic Encryption Marries Secret Sharing: Secure Large-Scale Sparse Logistic Regression and Applications in Risk Control.

Privacy-preserving semi-parallel logistic regression training with fully homomorphic encryption

Logistic regression model training based on the approximate homomorphic encryption

Privacy-Preserving CNN Training with Transfer Learning: Multiclass Logistic Regression

A Hybrid-Domain Framework for Secure Gradient Tree Boosting.

Efficient and Privacy-Preserving Logistic Regression Scheme Based on Leveled Fully Homomorphic Encryption

PrivLogit: Efficient Privacy-preserving Logistic Regression by Tailoring Numerical Optimizers

High Performance Logistic Regression for Privacy-Preserving Genome Analysis

LFFR: Logistic Function For (single-output) Regression

Privacy-preserving two-parties logistic regression on vertically partitioned data using asynchronous gradient sharing

Efficient Privacy Preserving Logistic Regression for Horizontally Distributed Data

Privacy-Preserving Locally Weighted Linear Regression over Encrypted Millions of Data.

Privacy-preserving model evaluation for logistic and linear regression using homomorphically encrypted genotype data

Privacy-Preserving Logistic Regression as a Cloud Service Based on Residue Number System.

Supporting Regularized Logistic Regression Privately and Efficiently

Efficient and Privacy-Preserving Logistic Regression Prediction over Vertically Partitioned Data

Privacy-Preserving Distributed Linear Regression on High-Dimensional Data

Collaborative Computing of Privacy-Preserving Logistic Regression Based on Homomorphic Encryption