Abstract:As a typical privacy-aware machine learning paradigm, federated learning (FL) provides facilities to individually train edge clients with their private data and aggregate the central global model. In this way, privacy leakage can be prevented. Massive communication overhead caused by exchanging updated weights between clients and the server is one of the main obstacles in this strategy. Prior work advocates compressing the weights by employing quantization, gradient sparsification, and knowledge distillation approaches. However, most of them cannot be readily applied to secure aggregation in privacy-aware FL. Some research has made great progress in directly utilizing benchmark secure aggregation protocols on top of the non-privacy-aware FL. Graph-based and gradient-based sparsification has been widely adopted in previous studies. However, the results of reducing communication costs are still unsatisfactory. In this paper, we present a novel communication-efficient privacy-ware FL algorithm from a distinct perspective. We design a new Two-Phase Sparsification with Secure Aggregation (TPSSA) algorithm. In the subnetwork phase, we identify sparse subnetworks by freezing the initial random weights in sufficiently overparametrized networks. All edge clients collaboratively train to discover their subnetwork inside a dense randomly weighted neural network. Then the server aggregates to compute the global model. In the gradient phase, for each pair of edge clients, we introduce pairwise multiplicative random masks to identify the sparsification pattern. Then updates from surviving clients can be correctly cancelled out during the aggregation process in the server. Theoretical analysis reveals convergence, privacy and performance guarantee. We show improvements in accuracy, communication, and computation over traditional and sparsified secure aggregation benchmarks on two real-world datasets.

Device Scheduling for Secure Aggregation in Wireless Federated Learning

Toward Secure and Private Over-the-Air Federated Learning

Device Scheduling for Relay-assisted Over-the-Air Aggregation in Federated Learning

Performance-Enhanced Federated Learning with Differential Privacy for Internet of Things

Privacy-preserving Aggregation Scheme for Blockchained Federated Learning in IoT.

Channel and Gradient-Importance Aware Device Scheduling for Over-the-Air Federated Learning

Scheduling Policies for Federated Learning in Wireless Networks:An Overview

Value of Information and Timing-aware Scheduling for Federated Learning

Low-Latency Federated Learning over Wireless Channels with Differential Privacy

Two-Phase Sparsification with Secure Aggregation for Privacy-Aware Federated Learning

Secure Weighted Aggregation for Federated Learning

Secure Aggregation in Heterogeneous Federated Learning for Digital Ecosystems

Mobility-Aware Joint User Scheduling and Resource Allocation for Low Latency Federated Learning

Device Scheduling with Fast Convergence for Wireless Federated Learning

Scheduling Policies for Federated Learning in Wireless Networks

Efficient Vertical Federated Learning with Secure Aggregation

Securing Secure Aggregation: Mitigating Multi-Round Privacy Leakage in Federated Learning

Reputation-Based Federated Learning for Secure Wireless Networks

User Scheduling for Federated Learning Through Over-the-Air Computation

Clustered Data Sharing for Non-IID Federated Learning over Wireless Networks

Intelligent Reflecting Surface-Assisted Low-Latency Federated Learning over Wireless Networks