Abstract:Differential privacy (DP), as a rigorous mathematical definition quantifying privacy leakage, has become a well-accepted standard for privacy protection. Combined with powerful machine learning (ML) techniques, differentially private machine learning (DPML) is increasingly important. As the most classic DPML algorithm, DP-SGD incurs a significant loss of utility, which hinders DPML's deployment in practice. Many studies have recently proposed improved algorithms based on DP-SGD to mitigate utility loss. However, these studies are isolated and cannot comprehensively measure the performance of improvements proposed in algorithms. More importantly, there is a lack of comprehensive research to compare improvements in these DPML algorithms across utility, defensive capabilities, and generalizability. We fill this gap by performing a holistic measurement of improved DPML algorithms on utility and defense capability against membership inference attacks (MIAs) on image classification tasks. We first present a taxonomy of where improvements are located in the ML life cycle. Based on our taxonomy, we jointly perform an extensive measurement study of the improved DPML algorithms, over twelve algorithms, four model architectures, four datasets, two attacks, and various privacy budget configurations. We also cover state-of-the-art label differential privacy (Label DP) algorithms in the evaluation. According to our empirical results, DP can effectively defend against MIAs, and sensitivity-bounding techniques such as per-sample gradient clipping play an important role in defense. We also explore some improvements that can maintain model utility and defend against MIAs more effectively. Experiments show that Label DP algorithms achieve less utility loss but are fragile to MIAs. ML practitioners may benefit from these evaluations to select appropriate algorithms. To support our evaluation, we implement a modular re-usable software, DPMLBench,(1) which enables sensitive data owners to deploy DPML algorithms and serves as a benchmark tool for researchers and practitioners.

ABDP: Accurate Billing on Differentially Private Data Reporting for Smart Grids

PPPA: A practical privacy-preserving aggregation scheme for smart grid communications

Data-Driven Optimization for Utility Providers with Differential Privacy of Users' Energy Profile

APED: An efficient aggregation protocol with error detection for smart grid communications.

A Decentralized Mechanism Based on Differential Privacy for Privacy-Preserving Computation in Smart Grid

Cost and Pricing of Differential Privacy in Demand Reporting for Smart Grids

When Privacy Meets Economics: Enabling Differentially-Private Battery-Supported Meter Reporting In Smart Grid

An Efficient Data Aggregation Scheme with Local Differential Privacy in Smart Grid

DPNCT: A Differential Private Noise Cancellation Scheme for Load Monitoring and Billing for Smart Meters

Achieving Differential Privacy of Data Disclosure in the Smart Grid.

E-DPNCT: An Enhanced Attack Resilient Differential Privacy Model For Smart Grids Using Split Noise Cancellation

Intermittently differential privacy in smart meters via rechargeable batteries

Optimization of social welfare and mitigating privacy risks in P2P energy trading: Differential privacy for secure data reporting

Local Differential Privacy for Smart Meter Data Sharing

Cost-Friendly Differential Privacy for Smart Meters: Exploiting the Dual Roles of the Noise

Differential-privacy Preserving Optimal Power Flow in Smart Grid

Differentially Private Demand Side Management for Incentivized Dynamic Pricing in Smart Grid

AdaPDP: Adaptive Personalized Differential Privacy.

Differentially Private Communication of Measurement Anomalies in the Smart Grid

Differentially Private Publication of Electricity Time Series Data in Smart Grids

DPMLBench: Holistic Evaluation of Differentially Private Machine Learning