Abstract:Machine Learning (ML), addresses a multitude of complex issues in multiple disciplines, including social sciences, finance, and medical research. ML models require substantial computing power and are only as powerful as the data utilized. Due to the high computational cost of ML methods, data scientists frequently use Machine Learning-as-a-Service (MLaaS) to outsource computation to external servers. However, when working with private information, like financial data or health records, outsourcing the computation might result in privacy issues. Recent advances in Privacy-Preserving Techniques (PPTs) have enabled ML training and inference over protected data through the use of Privacy-Preserving Machine Learning (PPML). However, these techniques are still at a preliminary stage and their application in real-world situations is demanding. In order to comprehend the discrepancy between theoretical research suggestions and actual applications, this work examines the past and present of PPML, focusing on Homomorphic Encryption (HE) and Secure Multi-party Computation (SMPC) applied to ML. This work primarily focuses on the ML model's training phase, where maintaining user data privacy is of utmost importance. We provide a solid theoretical background that eases the understanding of current approaches and their limitations. We also provide some preliminaries of SMPC, HE, and ML. In addition, we present a systemization of knowledge of the most recent PPML frameworks for model training and provide a comprehensive comparison in terms of the unique properties and performances on standard benchmarks. Also, we reproduce the results for some of the surveyed papers and examine at what level existing works in the field provide support for open science. We believe our work serves as a valuable contribution by raising awareness about the current gap between theoretical advancements and real-world applications in PPML, specifically regarding open-source availability, reproducibility, and usability.

Poster: Nebula: an Industrial-purpose Privacy-preserving Machine Learning System

Privacy-Preserving Collaborative Deep Learning with Unreliable Participants.

GELU-Net: A Globally Encrypted, Locally Unencrypted Deep Neural Network for Privacy-Preserved Learning.

CHEETAH: An Ultra-Fast, Approximation-Free, and Privacy-Preserved Neural Network Framework based on Joint Obscure Linear and Nonlinear Computations

Industrial Scale Privacy Preserving Deep Neural Network

A Distributed Privacy-Preserving Framework for Deep Learning with Edge-Cloud Computing.

SHAPER: A General Architecture for Privacy-Preserving Primitives in Secure Machine Learning.

SecureML: A System for Scalable Privacy-Preserving Machine Learning

Multiparty Secure Broad Learning System for Privacy Preserving

Privacy-Preserving Machine Learning Algorithms for Big Data Systems

Complete Security and Privacy for AI Inference in Decentralized Systems

Multi-party Secure Broad Learning System for Privacy Preserving

SSNet: A Lightweight Multi-Party Computation Scheme for Practical Privacy-Preserving Machine Learning Service in the Cloud

A secure distributed machine learning protocol against static semi-honest adversaries

Trusted AI in Multiagent Systems: An Overview of Privacy and Security for Distributed Learning

SoK: Wildest Dreams: Reproducible Research in Privacy-preserving Neural Network Training

Collaborative Machine Learning: Schemes, Robustness, and Privacy

Nebula-I: A General Framework for Collaboratively Training Deep Learning Models on Low-Bandwidth Cloud Clusters

Distributed Intelligent Model for Privacy and Secrecy in Preschool Education

A TEE-Based Federated Privacy Protection Method: Proposal and Implementation

Towards Secure and Practical Machine Learning Via Secret Sharing and Random Permutation