Abstract:Machine learning algorithms based on deep neural networks have achieved remarkable results and are being extensively used in different domains. However, the machine learning algorithms requires access to raw data which is often privacy sensitive. To address this issue, we develop new techniques to provide solutions for running deep neural networks over encrypted data. In this paper, we develop new techniques to adopt deep neural networks within the practical limitation of current homomorphic encryption schemes. More specifically, we focus on classification of the well-known convolutional neural networks (CNN). First, we design methods for approximation of the activation functions commonly used in CNNs (i.e. ReLU, Sigmoid, and Tanh) with low degree polynomials which is essential for efficient homomorphic encryption schemes. Then, we train convolutional neural networks with the approximation polynomials instead of original activation functions and analyze the performance of the models. Finally, we implement convolutional neural networks over encrypted data and measure performance of the models. Our experimental results validate the soundness of our approach with several convolutional neural networks with varying number of layers and structures. When applied to the MNIST optical character recognition tasks, our approach achieves 99.52\% accuracy which significantly outperforms the state-of-the-art solutions and is very close to the accuracy of the best non-private version, 99.77\%. Also, it can make close to 164000 predictions per hour. We also applied our approach to CIFAR-10, which is much more complex compared to MNIST, and were able to achieve 91.5\% accuracy with approximation polynomials used as activation functions. These results show that CryptoDL provides efficient, accurate and scalable privacy-preserving predictions.

Privacy-preserving Machine Learning as a Service

CHEETAH: An Ultra-Fast, Approximation-Free, and Privacy-Preserved Neural Network Framework based on Joint Obscure Linear and Nonlinear Computations

A Distributed Privacy-Preserving Framework for Deep Learning with Edge-Cloud Computing.

Industrial Scale Privacy Preserving Deep Neural Network

CryptoDL: Deep Neural Networks over Encrypted Data

SecureML: A System for Scalable Privacy-Preserving Machine Learning

2P-DNN : Privacy-Preserving Deep Neural Networks Based on Homomorphic Cryptosystem

A Privacy-Preserving Revocable Framework in the Deep-Learning-as-a-service Platform System Based on Non Software As a Service

Cryptographic Approaches for Privacy-Preserving Machine Learning

Privacy-Preserving Machine Learning As a Service: Challenges and Opportunities

GuardML: Efficient Privacy-Preserving Machine Learning Services Through Hybrid Homomorphic Encryption

Hyperdimensional Computing as a Rescue for Efficient Privacy-Preserving Machine Learning-as-a-Service

A Survey of Deep Learning Architectures for Privacy-Preserving Machine Learning With Fully Homomorphic Encryption

Privacy-Preserving Deep Learning on Machine Learning as a Service—a Comprehensive Survey

Privacy preserving Neural Network Inference on Encrypted Data with GPUs

Privacy Preserving Inference for Deep Neural Networks: Optimizing Homomorphic Encryption for Efficient and Secure Classification

A Pervasive, Efficient and Private Future: Realizing Privacy-Preserving Machine Learning Through Hybrid Homomorphic Encryption

A Novel Privacy-Preserving Neural Network Computing Approach for E-Health Information System

Efficient FHE-based Privacy-Enhanced Neural Network for Trustworthy AI-as-a-Service

MSCryptoNet: Multi-Scheme Privacy-Preserving Deep Learning in Cloud Computing.