Abstract:In recent years, there has been a growing interest in distributed optimization, which collaboratively attains an optimum by exchanging information with neighbours. Among the various distributed algorithms available, optimization with gradient tracking is particularly notable for its superior convergence results, especially in the context of directed graphs. However, privacy concerns arise when gradient information is transmitted directly which would induce more information leakage. Surprisingly, the literature has not adequately addressed the associated privacy issues. In response to the gap, our paper proposes a privacy-preserving distributed optimization algorithm with gradient tracking by adding noises to transmitted messages, namely, the decision variables and the estimate of the aggregated gradient. We prove two dilemmas for this kind of algorithm. In the first dilemma, we reveal that this distributed optimization algorithm with gradient tracking cannot achieve $ \epsilon$-differential privacy (DP) and exact convergence simultaneously. Building on this, we subsequently highlight that the algorithm fails to achieve $ \epsilon$-DP when employing non-summable stepsizes in the presence of Laplace noises. It is crucial to emphasize that these findings hold true regardless of the size of the privacy metric $ \epsilon$. After that, we rigorously analyse the convergence performance and privacy level given summable stepsize sequences under Laplace distribution since it is only with summable stepsizes that is meaningful for us to study. We derive sufficient conditions that allow for the simultaneous stochastically bounded accuracy and $ \epsilon$-DP. Recognizing that several options can meet these conditions, we further derive an upper bound of the mean error's variance and specify the mathematical expression of $ \epsilon$ under such conditions. Numerical simulations are provided to demonstrate the effectiveness of our proposed algorithm.

Efficient and Near-Optimal Noise Generation for Streaming Differential Privacy

An Algorithm for Streaming Differentially Private Data

Additive Noise Mechanisms for Making Randomized Approximation Algorithms Differentially Private

Differentially Private Continual Releases of Streaming Frequency Moment Estimations

Improved Pan-Private Stream Density Estimation

Optimum Noise Mechanism for Differentially Private Queries in Discrete Finite Sets

Streaming Submodular Maximization Under Differential Privacy Noise.

Noise Variance Optimization in Differential Privacy: A Game-Theoretic Approach Through Per-Instance Differential Privacy

Efficient Noise Generation Protocols for Differentially Private Multiparty Computation

Submodular Maximization over Data Streams with Differential Privacy Noise

Streaming Private Continual Counting via Binning

DPI: Ensuring Strict Differential Privacy for Infinite Data Streaming

Differential Privacy in Distributed Optimization with Gradient Tracking

Securely Sampling Discrete Gaussian Noise for Multi-Party Differential Privacy

Differentially private and decentralized randomized power method

A Differential Privacy Noise Dynamic Allocation Algorithm for Big Multimedia Data

A bounded-noise mechanism for differential privacy

Bounded Space Differentially Private Quantiles

Optimizing Noise for $f$-Differential Privacy via Anti-Concentration and Stochastic Dominance

Learning Numeric Optimal Differentially Private Truncated Additive Mechanisms