Abstract:Deep learning unlocks applications with societal impacts, e.g., detecting child exploitation imagery and genomic analysis of rare diseases. Deployment, however, needs compliance with stringent privacy regulations. Training algorithms that preserve the privacy of training data are in pressing need. Purely cryptographic approaches can protect privacy, but they are still costly, even when they rely on two or more non-colluding servers. Seemingly-"trivial" operations in plaintext quickly become prohibitively inefficient when a series of them are "crypto-processed," e.g., (dynamic) quantization for ensuring the intermediate values would not overflow. Slalom, recently proposed by Tramer and Boneh, is the first solution that leverages both GPU (for efficient batch computation) and a trusted execution environment (TEE) (for minimizing the use of cryptography). Roughly, it works by a lot of pre-computation over known and fixed weights, and hence it only supports private inference. Five related problems for private training are left unaddressed. Goten, our privacy-preserving training and prediction framework, tackles all five problems simultaneously via our careful design over the "mismatched" cryptographic and GPU data types (due to the tension between precision and efficiency) and our round-optimal GPU-outsourcing protocol (hence minimizing the communication cost between servers). It 1) stochastically trains a low-bitwidth yet accurate model, 2) supports dynamic quantization (a challenge left by Slalom), 3) minimizes the memory-swapping overhead of the memory-limited TEE and its communication with GPU, 4) crypto-protects the (dynamic) model weight from untrusted GPU, and 5) outperforms a pure-TEE system, even without pre-computation (needed by Slalom). As a baseline, we build CaffeScone that secures Caffe using TEE but not GPU; Goten shows a 6.84x speed-up of the whole VGG-11. Goten also outperforms Falcon proposed by Wagh et al., the latest secure multi-server cryptographic solution, by 132.64x using VGG-11. Lastly, we demonstrate Goten's efficacy in training models for breast cancer diagnosis over sensitive images.

Fregata: Fast Private Inference with Unified Secure Two-Party Protocols

CHEETAH: An Ultra-Fast, Approximation-Free, and Privacy-Preserved Neural Network Framework based on Joint Obscure Linear and Nonlinear Computations

Flash: A Hybrid Private Inference Protocol for Deep CNNs with High Accuracy and Low Latency on CPU

Toward Practical Privacy-Preserving Convolutional Neural Networks Exploiting Fully Homomorphic Encryption

FastSecNet: An Efficient Cryptographic Framework for Private Neural Network Inference

GFS-CNN: A GPU-friendly Secure Computation Platform for Convolutional Neural Networks

FPCNN: A fast privacy-preserving outsourced convolutional neural network with low-bandwidth

Optimized Privacy-Preserving CNN Inference With Fully Homomorphic Encryption

CryptoNite: Revealing the Pitfalls of End-to-End Private Inference at Scale

Towards Fast and Scalable Private Inference

C2PI: An Efficient Crypto-Clear Two-Party Neural Network Private Inference

PCNNCEC: Efficient and Privacy-Preserving Convolutional Neural Network Inference Based on Cloud-Edge-Client Collaboration

Dopamine receptor agonist reduces ethanol self-administration in the ethanol-preferring C57BL/6J inbred mouse.

FALCON: A Fourier Transform Based Approach for Fast and Secure Convolutional Neural Network Predictions

Gazelle: A Low Latency Framework for Secure Neural Network Inference

Privacy preserving Neural Network Inference on Encrypted Data with GPUs

DCT-CryptoNets: Scaling Private Inference in the Frequency Domain

Goten: GPU-Outsourcing Trusted Execution of Neural Network Training

A Secure Convolutional Neural Network Inference Model Based on Homomorphic Encryption

Efficient Privacy-Preserving Convolutional Spiking Neural Networks with FHE

Falcon: Accelerating Homomorphically Encrypted Convolutions for Efficient Private Mobile Network Inference