Abstract:Abstract Neural Networks (NN) provide a powerful method for machine learning training and inference. To effectively train, it is desirable for multiple parties to combine their data – however, doing so conflicts with data privacy. In this work, we provide novel three-party secure computation protocols for various NN building blocks such as matrix multiplication, convolutions, Rectified Linear Units, Maxpool, normalization and so on. This enables us to construct three-party secure protocols for training and inference of several NN architectures such that no single party learns any information about the data. Experimentally, we implement our system over Amazon EC2 servers in different settings. Our work advances the state-of-the-art of secure computation for neural networks in three ways: 1. Scalability: We are the first work to provide neural network training on Convolutional Neural Networks (CNNs) that have an accuracy of > 99% on the MNIST dataset; 2. Performance: For secure inference, our system outperforms prior 2 and 3-server works (SecureML, MiniONN, Chameleon, Gazelle) by 6×-113× (with larger gains obtained in more complex networks). Our total execution times are 2 − 4× faster than even just the online times of these works. For secure training, compared to the only prior work (SecureML) that considered a much smaller fully connected network, our protocols are 79× and 7× faster than their 2 and 3-server protocols. In the WAN setting, these improvements are more dramatic and we obtain an improvement of 553×! 3. Security: Our protocols provide two kinds of security: full security (privacy and correctness) against one semi-honest corruption and the notion of privacy against one malicious corruption [Araki et al . CCS’16]. All prior works only provide semi-honest security and ours is the first system to provide any security against malicious adversaries for the secure computation of complex algorithms such as neural network inference and training. Our gains come from a significant improvement in communication through the elimination of expensive garbled circuits and oblivious transfer protocols.

Securely and Efficiently Outsourcing Neural Network Inference via Parallel MSB Extraction.

CHEETAH: An Ultra-Fast, Approximation-Free, and Privacy-Preserved Neural Network Framework based on Joint Obscure Linear and Nonlinear Computations

Industrial Scale Privacy Preserving Deep Neural Network

FastSecNet: An Efficient Cryptographic Framework for Private Neural Network Inference

CrossNet: A Low-Latency MLaaS Framework for Privacy-Preserving Neural Network Inference on Resource-Limited Devices

Secure Prediction of Neural Network in the Cloud

Towards Scalable and Privacy-Preserving Deep Neural Network via Algorithmic-Cryptographic Co-design

FPCNN: A fast privacy-preserving outsourced convolutional neural network with low-bandwidth

Efficient Privacy-Preserving Inference Outsourcing for Convolutional Neural Networks.

Pratical Privacy-preserving Convolutional Neural Network Inference Framework with Edge Computing for Health Monitoring

Optimizing Privacy-Preserving Outsourced Convolutional Neural Network Predictions

Toward Scalable and Privacy-preserving Deep Neural Network Via Algorithmic-Cryptographic Co-design

ABNN(2): Secure Two-party Arbitrary-Bitwidth Quantized Neural Network Predictions

PCNNCEC: Efficient and Privacy-Preserving Convolutional Neural Network Inference Based on Cloud-Edge-Client Collaboration

Privacy-Preserving Computation Offloading for Parallel Deep Neural Networks Training

Joint Linear and Nonlinear Computation Across Functions for Efficient Privacy-Preserving Neural Network Inference

Communication-Efficient Privacy-Preserving Neural Network Inference via Arithmetic Secret Sharing

Private Inference for Deep Neural Networks: A Secure, Adaptive, and Efficient Realization

BAYHENN: Combining Bayesian Deep Learning and Homomorphic Encryption for Secure DNN Inference

FOBNN: Fast Oblivious Binarized Neural Network Inference

SecureNN: 3-Party Secure Computation for Neural Network Training