Abstract:Abstract Neural Networks (NN) provide a powerful method for machine learning training and inference. To effectively train, it is desirable for multiple parties to combine their data – however, doing so conflicts with data privacy. In this work, we provide novel three-party secure computation protocols for various NN building blocks such as matrix multiplication, convolutions, Rectified Linear Units, Maxpool, normalization and so on. This enables us to construct three-party secure protocols for training and inference of several NN architectures such that no single party learns any information about the data. Experimentally, we implement our system over Amazon EC2 servers in different settings. Our work advances the state-of-the-art of secure computation for neural networks in three ways: 1. Scalability: We are the first work to provide neural network training on Convolutional Neural Networks (CNNs) that have an accuracy of > 99% on the MNIST dataset; 2. Performance: For secure inference, our system outperforms prior 2 and 3-server works (SecureML, MiniONN, Chameleon, Gazelle) by 6×-113× (with larger gains obtained in more complex networks). Our total execution times are 2 − 4× faster than even just the online times of these works. For secure training, compared to the only prior work (SecureML) that considered a much smaller fully connected network, our protocols are 79× and 7× faster than their 2 and 3-server protocols. In the WAN setting, these improvements are more dramatic and we obtain an improvement of 553×! 3. Security: Our protocols provide two kinds of security: full security (privacy and correctness) against one semi-honest corruption and the notion of privacy against one malicious corruption [Araki et al . CCS’16]. All prior works only provide semi-honest security and ours is the first system to provide any security against malicious adversaries for the secure computation of complex algorithms such as neural network inference and training. Our gains come from a significant improvement in communication through the elimination of expensive garbled circuits and oblivious transfer protocols.

ABNN 2

ABNN(2): Secure Two-party Arbitrary-Bitwidth Quantized Neural Network Predictions

Privacy-Preserving Collaborative Deep Learning with Unreliable Participants.

CHEETAH: An Ultra-Fast, Approximation-Free, and Privacy-Preserved Neural Network Framework based on Joint Obscure Linear and Nonlinear Computations

Industrial Scale Privacy Preserving Deep Neural Network

Poster: Nebula: an Industrial-purpose Privacy-preserving Machine Learning System

SSNet: A Lightweight Multi-Party Computation Scheme for Practical Privacy-Preserving Machine Learning Service in the Cloud

An Efficient 3-Party Framework for Privacy-Preserving Neural Network Inference

Achieving Efficient and Privacy-Preserving Neural Network Training and Prediction in Cloud Environments

SecureML: A System for Scalable Privacy-Preserving Machine Learning

Secure Prediction of Neural Network in the Cloud

Practical Two-party Privacy-preserving Neural Network Based on Secret Sharing

SecureNN: 3-Party Secure Computation for Neural Network Training

Privacy Preserving Multi-Party Computation Delegation for Deep Learning in Cloud Computing

Empowering SMPC: Bridging the Gap Between Scalability, Memory Efficiency and Privacy in Neural Network Inference

CBNN: 3-Party Secure Framework for Customized Binary Neural Networks Inference

Secure Neural Network Prediction in the Cloud-Based Open Neural Network Service

NPMML: A Framework for Non-Interactive Privacy-Preserving Multi-Party Machine Learning

2P-DNN : Privacy-Preserving Deep Neural Networks Based on Homomorphic Cryptosystem

Secure Multi-Party Computation for Machine Learning: A Survey

Towards Scalable and Privacy-Preserving Deep Neural Network via Algorithmic-Cryptographic Co-design