Abstract:As a means of decentralized machine learning, federated learning (FL) has recently drawn considerable attentions. One of the prominent advantages of FL is its capability of preventing clients' data from being directly exposed to external adversaries. Nevertheless, via a viewpoint of information theory, it is still possible for an attacker to steal private information from eavesdropping upon the shared models uploaded by FL clients. In order to address this problem, we develop a novel privacy preserving FL framework based on the concept of differential privacy (DP). To be specific, we first borrow the concept of local DP and introduce a client-level DP (CDP) by adding artificial noises to the shared models before uploading them to servers. Then, we prove that our proposed CDP algorithm can satisfy the DP guarantee with adjustable privacy protection levels by varying the variances of the artificial noises. More importantly, we derive a theoretical convergence upper-bound of the CDP algorithm. Our derived upper-bound reveals that there exists an optimal number of communication rounds to achieve the best convergence performance in terms of loss function values for a given privacy protection level. Furthermore, to obtain this optimal number of communication rounds, which cannot be derived in a closed-form expression, we propose a communication rounds discounting (CRD) method. Compared with the heuristic searching method, our proposed CRD can achieve a much better trade-off between the computational complexity of searching for the optimal number and the convergence performance. Extensive experiments indicate that our CDP algorithm with an optimization on the number of communication rounds using the proposed CRD can effectively improve both the FL training efficiency and FL model quality for a given privacy protection level.

How to Privately Tune Hyperparameters in Federated Learning? Insights from a Benchmark Study

Projected Federated Averaging with Heterogeneous Differential Privacy.

Privately Customizing Prefinetuning to Better Match User Data in Federated Learning

Federated Learning Hyper-Parameter Tuning From A System Perspective

Balancing Privacy and Performance for Private Federated Learning Algorithms

Performance Analysis and Optimization in Privacy-Preserving Federated Learning

Toward the Tradeoffs between Privacy, Fairness and Utility in Federated Learning

Fair Federated Learning with Multi-Objective Hyperparameter Optimization

When Federated Learning Meets Pre-trained Language Models' Parameter-Efficient Tuning Methods

Immersion and Invariance-based Coding for Privacy-Preserving Federated Learning

Provable Mutual Benefits from Federated Learning in Privacy-Sensitive Domains

Personalized Privacy-Preserving Federated Learning: Optimized Trade-off Between Utility and Privacy

Privacy-Preserving Federated Learning with Differentially Private Hyperdimensional Computing

Privacy-preserving gradient-based fair federated learning

Efficient federated learning privacy preservation method with heterogeneous differential privacy

On the Practicality of Differential Privacy in Federated Learning by Tuning Iteration Times

Hyper-parameter Optimization for Federated Learning with Step-wise Adaptive Mechanism

Utility-Aware Privacy-Preserving Federated Learning Through Information Bottleneck

Sparse Federated Learning with Hierarchical Personalized Models

A New Implementation of Federated Learning for Privacy and Security Enhancement

A Stochastic Optimization Framework for Private and Fair Learning From Decentralized Data