Abstract:Background: Security concerns have been raised since big data became a prominent tool in data analysis. For instance, many machine learning algorithms aim to generate prediction models using training data which contain sensitive information about individuals. Cryptography community is considering secure computation as a solution for privacy protection. In particular, practical requirements have triggered research on the efficiency of cryptographic primitives. Methods: This paper presents a method to train a logistic regression model without information leakage. We apply the homomorphic encryption scheme of Cheon et al. (ASIACRYPT 2017) for an efficient arithmetic over real numbers, and devise a new encoding method to reduce storage of encrypted database. In addition, we adapt Nesterov's accelerated gradient method to reduce the number of iterations as well as the computational cost while maintaining the quality of an output classifier. Results: Our method shows a state-of-the-art performance of homomorphic encryption system in a real-world application. The submission based on this work was selected as the best solution of Track 3 at iDASH privacy and security competition 2017. For example, it took about six minutes to obtain a logistic regression model given the dataset consisting of 1579 samples, each of which has 18 features with a binary outcome variable. Conclusions: We present a practical solution for outsourcing analysis tools such as logistic regression analysis while preserving the data confidentiality.

Privacy-Preserving Logistic Regression Training with A Faster Gradient Variant

Privacy-Preserving Logistic Regression Training on Large Datasets

Multinomial Logistic Regression Algorithms via Quadratic Gradient

PrivLogit: Efficient Privacy-preserving Logistic Regression by Tailoring Numerical Optimizers

Faster Differentially Private Convex Optimization via Second-Order Methods

On the Privacy of Noisy Stochastic Gradient Descent for Convex Optimization

A(DP)$^2$SGD: Asynchronous Decentralized Parallel Stochastic Gradient Descent with Differential Privacy

Improving Differentially Private SGD via Randomly Sparsified Gradients

A(DP)$^2$2SGD: Asynchronous Decentralized Parallel Stochastic Gradient Descent with Differential Privacy

DP-LSSGD: A Stochastic Optimization Method to Lift the Utility in Privacy-Preserving ERM

Quadratic Gradient: Combining Gradient Algorithms and Newton's Method as One

Homomorphically encrypted gradient descent algorithms for quadratic programming

Logistic regression model training based on the approximate homomorphic encryption

High Performance Logistic Regression for Privacy-Preserving Genome Analysis

Privacy for Free in the Over-Parameterized Regime

SSGD: A Safe and Efficient Method of Gradient Descent

Concentrated Differentially Private Gradient Descent with Adaptive per-Iteration Privacy Budget

Privacy Loss of Noisy Stochastic Gradient Descent Might Converge Even for Non-Convex Losses

An Improved Privacy-Preserving Stochastic Gradient Descent Algorithm

Large Stepsize Gradient Descent for Logistic Loss: Non-Monotonicity of the Loss Improves Optimization Efficiency