Abstract:There is a significant recent interest in collaboratively training a machine learning (ML) model without collecting data to a central server. Federated learning (FL) emerges as an efficient solution mitigating systemic privacy risks and communication costs. However, conventional FL inherited from parameter server designs relies too much on a central server, which may lead to privacy risks, communication bottlenecks, or a single point of failure. In this paper, we propose an asynchronous and hierarchical local gradient aggregation and global model update algorithm, FedDual, under three different security considerations for FL in large decentralized networks. Particularly, FedDual preserves privacy by introducing local differential privacy (LDP) and aggregates local gradients asynchronously and hierarchically via a pair-wise gossip algorithm, which is more competitive than previous gossip-based decentralized FL methods in terms of privacy preservation and communication efficiency, and offers more computational efficiency compared to existing blockchain-assisted decentralized FL methods. Further, we devise a noise cutting trick based on Private Set Intersection (PSI) to mitigate the prediction performance loss of the global model caused by the leveraged LDP. Rigorous analyses show that FedDual helps decentralized FL achieve the same convergence rate of $\mathcal {O}\left({\frac {1}{T}}\right) $ as centralized ML theoretically. Ingenious experiments on MNIST, CIFAR-10, and FEMNIST confirm that the model prediction performance gained from FedDual is close to centralized ML. More importantly, the proposed noise cutting trick helps FedDual to train better global models than LDP-based FL methods in terms of prediction performance and convergence rate.

Improving Model Consistency of Decentralized Federated Learning via Sharpness Aware Minimization and Multiple Gossip Approaches

Improving the Model Consistency of Decentralized Federated Learning

DFedADMM: Dual Constraints Controlled Model Inconsistency for Decentralized Federated Learning

Decentralized Federated Learning under Communication Delays.

Dynamic Regularized Sharpness Aware Minimization in Federated Learning: Approaching Global Consistency and Smooth Landscape

Decentralized Sporadic Federated Learning: A Unified Algorithmic Framework with Convergence Guarantees

Towards the Flatter Landscape and Better Generalization in Federated Learning under Client-level Differential Privacy

Decentralized Federated Learning: Balancing Communication and Computing Costs

GossipFL: A Decentralized Federated Learning Framework with Sparsified and Adaptive Communication

Locally Estimated Global Perturbations are Better than Local Perturbations for Federated Sharpness-aware Minimization

Enhancing Federated Learning Convergence with Dynamic Data Queue and Data Entropy-driven Participant Selection

FedDual: Pair-Wise Gossip Helps Federated Learning in Large Decentralized Networks

FedGAMMA: Federated Learning with Global Sharpness-Aware Minimization.

Privacy-Preserving and Reliable Decentralized Federated Learning

Robust Model Aggregation for Heterogeneous Federated Learning: Analysis and Optimizations

On the Convergence of Communication-Efficient Local SGD for Federated Learning

Beyond Local Sharpness: Communication-Efficient Global Sharpness-aware Minimization for Federated Learning

On the Convergence of Decentralized Federated Learning Under Imperfect Information Sharing

Make Landscape Flatter in Differentially Private Federated Learning

Towards Efficient Federated Learning Framework Via Selective Aggregation of Models

General Decentralized Federated Learning for Communication-Computation Tradeoff.