Abstract:Abstract Neural Networks (NN) provide a powerful method for machine learning training and inference. To effectively train, it is desirable for multiple parties to combine their data – however, doing so conflicts with data privacy. In this work, we provide novel three-party secure computation protocols for various NN building blocks such as matrix multiplication, convolutions, Rectified Linear Units, Maxpool, normalization and so on. This enables us to construct three-party secure protocols for training and inference of several NN architectures such that no single party learns any information about the data. Experimentally, we implement our system over Amazon EC2 servers in different settings. Our work advances the state-of-the-art of secure computation for neural networks in three ways: 1. Scalability: We are the first work to provide neural network training on Convolutional Neural Networks (CNNs) that have an accuracy of > 99% on the MNIST dataset; 2. Performance: For secure inference, our system outperforms prior 2 and 3-server works (SecureML, MiniONN, Chameleon, Gazelle) by 6×-113× (with larger gains obtained in more complex networks). Our total execution times are 2 − 4× faster than even just the online times of these works. For secure training, compared to the only prior work (SecureML) that considered a much smaller fully connected network, our protocols are 79× and 7× faster than their 2 and 3-server protocols. In the WAN setting, these improvements are more dramatic and we obtain an improvement of 553×! 3. Security: Our protocols provide two kinds of security: full security (privacy and correctness) against one semi-honest corruption and the notion of privacy against one malicious corruption [Araki et al . CCS’16]. All prior works only provide semi-honest security and ours is the first system to provide any security against malicious adversaries for the secure computation of complex algorithms such as neural network inference and training. Our gains come from a significant improvement in communication through the elimination of expensive garbled circuits and oblivious transfer protocols.

FALCON: Honest-Majority Maliciously Secure Framework for Private Deep Learning

CHEETAH: An Ultra-Fast, Approximation-Free, and Privacy-Preserved Neural Network Framework based on Joint Obscure Linear and Nonlinear Computations

GELU-Net: A Globally Encrypted, Locally Unencrypted Deep Neural Network for Privacy-Preserved Learning.

Falcon: A Privacy-Preserving and Interpretable Vertical Federated Learning System

FALCON: A Fourier Transform Based Approach for Fast and Secure Convolutional Neural Network Predictions

Falcon: Accelerating Homomorphically Encrypted Convolutions for Efficient Private Mobile Network Inference

FLEDGE: Ledger-based Federated Learning Resilient to Inference and Backdoor Attacks

SSNet: A Lightweight Multi-Party Computation Scheme for Practical Privacy-Preserving Machine Learning Service in the Cloud

Secure and Efficient Decentralized Federated Learning with Data Representation Protection

Efficient, Private and Robust Federated Learning

Secure and Efficient Federated Transfer Learning

SecureNN: 3-Party Secure Computation for Neural Network Training

FastSecNet: An Efficient Cryptographic Framework for Private Neural Network Inference

Efficient Privacy-Preserving Machine Learning with Lightweight Trusted Hardware

SPEED: Secure, PrivatE, and Efficient Deep learning

HybridAlpha: An Efficient Approach for Privacy-Preserving Federated Learning

Privacy preserving layer partitioning for Deep Neural Network models

Flamingo: Multi-Round Single-Server Secure Aggregation with Applications to Private Federated Learning

FssNN: Communication-Efficient Secure Neural Network Training via Function Secret Sharing

Privacy-Preserving Federated Learning Framework Based on Chained Secure Multiparty Computing

All Rivers Run to the Sea: Private Learning with Asymmetric Flows