Abstract:The randomized power method has gained significant interest due to its simplicity and efficient handling of large-scale spectral analysis and recommendation tasks. As modern datasets contain sensitive private information, we need to give formal guarantees on the possible privacy leaks caused by this method. This paper focuses on enhancing privacy preserving variants of the method. We propose a strategy to reduce the variance of the noise introduced to achieve Differential Privacy (DP). We also adapt the method to a decentralized framework with a low computational and communication overhead, while preserving the accuracy. We leverage Secure Aggregation (a form of Multi-Party Computation) to allow the algorithm to perform computations using data distributed among multiple users or devices, without revealing individual data. We show that it is possible to use a noise scale in the decentralized setting that is similar to the one in the centralized setting. We improve upon existing convergence bounds for both the centralized and decentralized versions. The proposed method is especially relevant for decentralized applications such as distributed recommender systems, where privacy concerns are paramount.

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is the lack of privacy protection in the existing Randomized Power Method when dealing with large - scale spectral analysis and recommendation tasks. Specifically: 1. **Privacy Leakage Problem**: Modern datasets usually contain sensitive personal information, and the traditional Randomized Power Method cannot provide sufficient privacy protection. Therefore, a formal privacy - protection mechanism needs to be introduced to ensure that the algorithm output does not leak too much individual information. 2. **Noise Introduction and Performance Balance**: In order to achieve Differential Privacy (DP), noise needs to be introduced into the calculation process. However, the introduction of noise in existing methods will lead to a large variance, thus affecting the accuracy and efficiency of the algorithm. This paper proposes a strategy to reduce the noise variance introduced to achieve differential privacy, so as to improve the algorithm performance. 3. **Application in Decentralized Environments**: In decentralized environments, such as distributed recommendation systems, data is distributed among multiple users or devices, and communication is limited by a predefined communication graph. Most of the existing methods rely on a centralized setting, that is, assuming that there is a trusted manager holding all data, which is not always feasible in practical applications. This paper proposes a method suitable for the decentralized framework, and performs calculations without exposing personal data through techniques such as Secure Aggregation. 4. **Convergence Analysis**: Existing methods have limitations in terms of the number of singular vectors calculated, data distribution assumptions, etc., which lead to limitations in their performance and privacy protection. This paper improves the convergence bound, reduces the sensitivity to the number of singular vectors, and provides new privacy proofs, correcting the errors in previous works. In summary, this paper aims to improve the privacy and computational efficiency of the Randomized Power Method when dealing with large - scale data by introducing an improved privacy - protection mechanism and a method suitable for the decentralized environment.

Differentially private and decentralized randomized power method

Differentially Private Distributed Estimation and Learning

Additive Noise Mechanisms for Making Randomized Approximation Algorithms Differentially Private

Differentially Private Distributed Resource Allocation Via Deviation Tracking

A(DP)$^2$SGD: Asynchronous Decentralized Parallel Stochastic Gradient Descent with Differential Privacy

A(DP)$^2$2SGD: Asynchronous Decentralized Parallel Stochastic Gradient Descent with Differential Privacy

Optimum Noise Mechanism for Differentially Private Queries in Discrete Finite Sets

Efficient and Near-Optimal Noise Generation for Streaming Differential Privacy

Customized Local Differential Privacy for Multi-Agent Distributed Optimization

Fair and Differentially Private Distributed Frequency Estimation

Differentially Private Algorithms for Synthetic Power System Datasets

The Normal Distributions Indistinguishability Spectrum and its Application to Privacy-Preserving Machine Learning

Distributed empirical risk minimization with differential privacy

Privacy-Preserving Distributed Optimisation using Stochastic PDMM

The Privacy Power of Correlated Noise in Decentralized Learning

Improving Differentially Private SGD via Randomly Sparsified Gradients

When Differential Privacy Meets Randomized Perturbation: A Hybrid Approach For Privacy-Preserving Recommender System

Muffliato: Peer-to-Peer Privacy Amplification for Decentralized Optimization and Averaging

From Noisy Fixed-Point Iterations to Private ADMM for Centralized and Federated Learning

Differentially Private Distributed Nonconvex Stochastic Optimization with Quantized Communication

FedPower: Privacy-Preserving Distributed Eigenspace Estimation