Abstract:There is a significant recent interest in collaboratively training a machine learning (ML) model without collecting data to a central server. Federated learning (FL) emerges as an efficient solution mitigating systemic privacy risks and communication costs. However, conventional FL inherited from parameter server designs relies too much on a central server, which may lead to privacy risks, communication bottlenecks, or a single point of failure. In this paper, we propose an asynchronous and hierarchical local gradient aggregation and global model update algorithm, FedDual, under three different security considerations for FL in large decentralized networks. Particularly, FedDual preserves privacy by introducing local differential privacy (LDP) and aggregates local gradients asynchronously and hierarchically via a pair-wise gossip algorithm, which is more competitive than previous gossip-based decentralized FL methods in terms of privacy preservation and communication efficiency, and offers more computational efficiency compared to existing blockchain-assisted decentralized FL methods. Further, we devise a noise cutting trick based on Private Set Intersection (PSI) to mitigate the prediction performance loss of the global model caused by the leveraged LDP. Rigorous analyses show that FedDual helps decentralized FL achieve the same convergence rate of $\mathcal {O}\left({\frac {1}{T}}\right) $ as centralized ML theoretically. Ingenious experiments on MNIST, CIFAR-10, and FEMNIST confirm that the model prediction performance gained from FedDual is close to centralized ML. More importantly, the proposed noise cutting trick helps FedDual to train better global models than LDP-based FL methods in terms of prediction performance and convergence rate.

GossipFL: A Decentralized Federated Learning Framework with Sparsified and Adaptive Communication

Decentralized Federated Learning under Communication Delays.

Secure and Efficient Decentralized Federated Learning with Data Representation Protection

GFL: A Decentralized Federated Learning Framework Based On Blockchain.

FedDual: Pair-Wise Gossip Helps Federated Learning in Large Decentralized Networks

Improving Model Consistency of Decentralized Federated Learning via Sharpness Aware Minimization and Multiple Gossip Approaches

BlockDFL: A Blockchain-based Fully Decentralized Peer-to-Peer Federated Learning Framework

Decentralized Sporadic Federated Learning: A Unified Algorithmic Framework with Convergence Guarantees

Improving the Model Consistency of Decentralized Federated Learning

Communication-Efficient Federated Learning: A Variance-Reduced Stochastic Approach with Adaptive Sparsification.

Dynamic Attention-based Communication-Efficient Federated Learning

On the Convergence of Communication-Efficient Local SGD for Federated Learning

FedGK: Communication-Efficient Federated Learning through Group-Guided Knowledge Distillation

Privacy-Preserving and Reliable Decentralized Federated Learning

Decentralized Federated Learning: A Segmented Gossip Approach

AEDFL: Efficient Asynchronous Decentralized Federated Learning with Heterogeneous Devices

A Fair Federated Learning Framework with Reinforcement Learning.

Adaptive Control of Client Selection and Gradient Compression for Efficient Federated Learning

Heterogeneity-Aware Federated Learning with Adaptive Client Selection and Gradient Compression.

Efficient and Less Centralized Federated Learning

Communication-and-Computation Efficient Split Federated Learning: Gradient Aggregation and Resource Management