A Game-Theoretic Analysis of Auditing Differentially Private Algorithms with Epistemically Disparate Herd

Ya-Ting Yang,Tao Zhang,Quanyan Zhu

2024-04-25

Abstract:Privacy-preserving AI algorithms are widely adopted in various domains, but the lack of transparency might pose accountability issues. While auditing algorithms can address this issue, machine-based audit approaches are often costly and time-consuming. Herd audit, on the other hand, offers an alternative solution by harnessing collective intelligence. Nevertheless, the presence of epistemic disparity among auditors, resulting in varying levels of expertise and access to knowledge, may impact audit performance. An effective herd audit will establish a credible accountability threat for algorithm developers, incentivizing them to uphold their claims. In this study, our objective is to develop a systematic framework that examines the impact of herd audits on algorithm developers using the Stackelberg game approach. The optimal strategy for auditors emphasizes the importance of easy access to relevant information, as it increases the auditors' confidence in the audit process. Similarly, the optimal choice for developers suggests that herd audit is viable when auditors face lower costs in acquiring knowledge. By enhancing transparency and accountability, herd audit contributes to the responsible development of privacy-preserving algorithms.

Cryptography and Security,Computer Science and Game Theory

What problem does this paper attempt to address?

The problem this paper attempts to address is the audit issue of privacy-preserving algorithms. Specifically, the paper explores how to improve the transparency and accountability of algorithms through herd audit, particularly for differential privacy algorithms. Faced with the responsibility issues brought by the algorithm black box, traditional machine audit methods are often costly and time-consuming, while herd audit provides an alternative solution by leveraging collective intelligence. However, the knowledge differences among auditors may lead to varying audit effectiveness. Therefore, the goal of the paper is to construct a system framework that uses Stackelberg game theory to study the impact of herd audit on algorithm developers and to find the optimal strategy to enhance audit efficiency and credibility. When the cost of obtaining information for auditors is low, herd audit is more effective, which can encourage algorithm developers to adhere to their commitments, thereby promoting the responsible development of privacy-preserving algorithms.

A Game-Theoretic Analysis of Auditing Differentially Private Algorithms with Epistemically Disparate Herd

Debugging Differential Privacy: A Case Study for Privacy Auditing

Tight Auditing of Differentially Private Machine Learning

A General Framework for Auditing Differentially Private Machine Learning

Differentially Private Nash Equilibrium Seeking in Quadratic Network Games

Audit Games

Curator Attack: When Blackbox Differential Privacy Auditing Loses Its Power

Differentially-private Distributed Algorithms for Aggregative Games with Guaranteed Convergence

A Privacy Protection Model of Data Publication Based on Game Theory

Privacy-Accuracy Trade-Off in Differentially-Private Distributed Classification: A Game Theoretical Approach

Auditing $f$-Differential Privacy in One Run

Two-Party Privacy Games: How Users Perturb When Learners Preempt

Auditing Private Prediction

"What do you want from theory alone?" Experimenting with Tight Auditing of Differentially Private Synthetic Data Generation

Differential Privacy for the Analyst via Private Equilibrium Computation

The Last Iterate Advantage: Empirical Auditing and Principled Heuristic Analysis of Differentially Private SGD

Auditing Differential Privacy Guarantees Using Density Estimation

Unleashing the Power of Randomization in Auditing Differentially Private ML

Peer-induced Fairness: A Causal Approach for Algorithmic Fairness Auditing

Audit Games with Multiple Defender Resources