Abstract:As a means of decentralized machine learning, federated learning (FL) has recently drawn considerable attentions. One of the prominent advantages of FL is its capability of preventing clients' data from being directly exposed to external adversaries. Nevertheless, via a viewpoint of information theory, it is still possible for an attacker to steal private information from eavesdropping upon the shared models uploaded by FL clients. In order to address this problem, we develop a novel privacy preserving FL framework based on the concept of differential privacy (DP). To be specific, we first borrow the concept of local DP and introduce a client-level DP (CDP) by adding artificial noises to the shared models before uploading them to servers. Then, we prove that our proposed CDP algorithm can satisfy the DP guarantee with adjustable privacy protection levels by varying the variances of the artificial noises. More importantly, we derive a theoretical convergence upper-bound of the CDP algorithm. Our derived upper-bound reveals that there exists an optimal number of communication rounds to achieve the best convergence performance in terms of loss function values for a given privacy protection level. Furthermore, to obtain this optimal number of communication rounds, which cannot be derived in a closed-form expression, we propose a communication rounds discounting (CRD) method. Compared with the heuristic searching method, our proposed CRD can achieve a much better trade-off between the computational complexity of searching for the optimal number and the convergence performance. Extensive experiments indicate that our CDP algorithm with an optimization on the number of communication rounds using the proposed CRD can effectively improve both the FL training efficiency and FL model quality for a given privacy protection level.

Joint Communication and Learning Design of Differential Privacy for Federated Learning over Multi-cell Networks.

Projected Federated Averaging with Heterogeneous Differential Privacy.

On differential privacy for federated learning in wireless systems with multiple base stations

Performance-Enhanced Federated Learning with Differential Privacy for Internet of Things

Differential Privacy Meets Federated Learning under Communication Constraints

Differentially Private Over-the-Air Federated Learning Over MIMO Fading Channels

Performance Analysis and Optimization in Privacy-Preserving Federated Learning

Differentially-Private Multi-Tier Federated Learning

Joint User Association and Resource Allocation for Wireless Hierarchical Federated Learning with IID and Non-IID Data

Hierarchical federated learning with global differential privacy

Collaboration in Federated Learning With Differential Privacy: A Stackelberg Game Analysis

A Communication-Efficient, Privacy-Preserving Federated Learning Algorithm Based on Two-Stage Gradient Pruning and Differentiated Differential Privacy

Gradient sparsification for efficient wireless federated learning with differential privacy

Energy Efficient and Differentially Private Federated Learning via a Piggyback Approach

Communication-Efficient and Privacy-Preserving Aggregation in Federated Learning With Adaptability

A Theoretical Analysis of Efficiency Constrained Utility-Privacy Bi-Objective Optimization in Federated Learning

Privacy As a Resource in Differentially Private Federated Learning.

QI-DPFL: Quality-Aware and Incentive-Boosted Federated Learning with Differential Privacy

Federated learning with differential privacy via fast Fourier transform for tighter-efficient combining

A Two-Stage Differential Privacy Scheme for Federated Learning Based on Edge Intelligence