Abstract:Federated Learning (FL) requires frequent exchange of model parameters, which leads to long communication delay, especially when the network environments of clients vary greatly. Moreover, the parameter server needs to wait for the slowest client (i.e., straggler, which may have the largest model size, lowest computing capability or worst network condition) to upload parameters, which may significantly degrade the communication efficiency. Commonly-used client selection methods such as partial client selection would lead to the waste of computing resources and weaken the generalization of the global model. To tackle this problem, along a different line, in this paper, we advocate the approach of model parameter dropout instead of client selection, and accordingly propose a novel framework of Federated learning scheme with Differential parameter Dropout (FedDD). FedDD consists of two key modules: dropout rate allocation and uploaded parameter selection, which will optimize the model parameter uploading ratios tailored to different clients' heterogeneous conditions and also select the proper set of important model parameters for uploading subject to clients' dropout rate constraints. Specifically, the dropout rate allocation is formulated as a convex optimization problem, taking system heterogeneity, data heterogeneity, and model heterogeneity among clients into consideration. The uploaded parameter selection strategy prioritizes on eliciting important parameters for uploading to speedup convergence. Furthermore, we theoretically analyze the convergence of the proposed FedDD scheme. Extensive performance evaluations demonstrate that the proposed FedDD scheme can achieve outstanding performances in both communication efficiency and model convergence, and also possesses a strong generalization capability to data of rare classes.

DROPFL: Client Dropout Attacks Against Federated Learning Under Communication Constraints.

Secure Vertical Federated Learning Under Unreliable Connectivity

FedDD: Toward Communication-efficient Federated Learning with Differential Parameter Dropout

Secure and Efficient Decentralized Federated Learning with Data Representation Protection

DTPP-DFL: A Dropout-Tolerated Privacy-Preserving Decentralized Federated Learning Framework

Federated Learning With Unreliable Clients: Performance Analysis and Mechanism Design

Adaptive Federated Dropout: Improving Communication Efficiency and Generalization for Federated Learning

Dordis: Efficient Federated Learning with Dropout-Resilient Differential Privacy

Network-Level Adversaries in Federated Learning

Privacy-Preserving and Reliable Decentralized Federated Learning

Federated Dropout -- A Simple Approach for Enabling Federated Learning on Resource Constrained Devices

Differentially Private Federated Learning: Algorithm, Analysis and Optimization

Towards Practical Backdoor Attacks on Federated Learning Systems

On the Vulnerability of Backdoor Defenses for Federated Learning

FedBIAD: Communication-Efficient and Accuracy-Guaranteed Federated Learning with Bayesian Inference-Based Adaptive Dropout

Adaptive Selection of Loss Function for Federated Learning Clients Under Adversarial Attacks

Towards Stagewise and Energy-Accuracy-balanced Client Selection and Resource Allocation over Dynamic Federated Learning Networks

Lightweight Federated Learning with Differential Privacy and Straggler Resilience

Towards Understanding Adversarial Transferability in Federated Learning

DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Training

Robust Federated Learning Mitigates Client-side Training Data Distribution Inference Attacks