Abstract:Federated learning (FL) enables distributed clients to collaboratively train a machine learning model without sharing raw data with each other. However, it suffers from the leakage of private information from uploading models. In addition, as the model size grows, the training latency increases due to the limited transmission bandwidth and model performance degradation while using differential privacy (DP) protection. In this paper, we propose a gradient sparsification empowered FL framework with DP over wireless channels, to improve training efficiency without sacrificing convergence performance. Specifically, we first design a random sparsification algorithm to retain a fraction of the gradient elements in each client's local model, thereby mitigating the performance degradation induced by DP and reducing the number of transmission parameters over wireless channels. Then, we analyze the convergence bound of the proposed algorithm, by modeling a non-convex FL problem. Next, we formulate a time-sequential stochastic optimization problem for minimizing the developed convergence bound, under the constraints of transmit power, the average transmitting delay, as well as the client's DP requirement. Utilizing the Lyapunov drift-plus-penalty framework, we develop an analytical solution to the optimization problem. Extensive experiments have been implemented on three real-life datasets to demonstrate the effectiveness of our proposed algorithm. We show that our proposed algorithms can fully exploit the interworking between communication and computation to outperform the baselines, i.e., random scheduling, round robin, and delay-minimization algorithms.

Communication-Efficient Diffusion Strategy for Performance Improvement of Federated Learning with Non-IID Data

Over-the-air Learning Rate Optimization for Federated Learning

Optimizing Federated Learning on Non-IID Data Using Local Shapley Value.

Federated Learning with Additional Mechanisms on Clients to Reduce Communication Costs

Hierarchical Federated Learning with Adaptive Clustering on Non-IID Data

Communication-efficient federated continual learning for distributed learning system with Non-IID data

Client Selection for Federated Learning With Non-IID Data in Mobile Edge Computing

Decentralized Federated Learning with Data Feature Transmission and Neighbor Selection.

Data Collaborative Federated Learning for Non-i.i.d Data in Wireless Networks

Decentralized Federated Learning with Intermediate Results in Mobile Edge Computing

Communication-Efficient Federated Data Augmentation on Non-IID Data

Enhancing Federated Learning Convergence with Dynamic Data Queue and Data Entropy-driven Participant Selection

User-Distribution-Aware Federated Learning for Efficient Communication and Fast Inference

Communication-Efficient and Privacy-Preserving Aggregation in Federated Learning With Adaptability

FedRelay: Federated Relay Learning for 6G Mobile Edge Intelligence

D2D-Assisted Federated Learning in Mobile Edge Computing Networks

Communication-Efficient Model Aggregation with Layer Divergence Feedback in Federated Learning

An Optimization Method for Non-IID Federated Learning Based on Deep Reinforcement Learning

Communication-Efficient Federated Learning in Channel Constrained Internet of Things

Clustered Data Sharing for Non-IID Federated Learning over Wireless Networks

Gradient sparsification for efficient wireless federated learning with differential privacy