Abstract:Most existing wireless federated learning (FL) studies focused on homogeneous model settings where devices train identical local models. In this setting, the devices with poor communication and computation capabilities may delay the global model update and degrade the performance of FL. Moreover, in the homogenous model settings, the scale of the global model is restricted by the device with the lowest capability. To tackle these challenges, this work proposes an adaptive model pruning-based FL (AMP-FL) framework, where the edge server dynamically generates sub-models by pruning the global model for devices’ local training to adapt their heterogeneous computation capabilities and time-varying channel conditions. Since the involvement of diverse structures of devices’ sub-models in the global model updating may negatively affect the training convergence, we propose compensating for the gradients of pruned model regions by devices’ historical gradients. We then introduce an age of information (AoI) metric to characterize the staleness of local gradients and theoretically analyze the convergence behaviour of AMP-FL. The convergence bound suggests scheduling devices with large AoI of gradients and pruning the model regions with small AoI for devices to improve the learning performance. Inspired by this, we define a new objective function, i.e., the average AoI of local gradients, to transform the inexplicit global loss minimization problem into a tractable one for device scheduling, model pruning, and resource block (RB) allocation design. Through detailed analysis, we derive the optimal model pruning strategy and transform the RB allocation problem into equivalent linear programming that can be effectively solved. Experimental results demonstrate the effectiveness and superiority of the proposed approaches. The proposed AMP-FL is capable of achieving 1.9x and 1.6x speed up for FL on MNIST and CIFAR-10 datasets in comparison with the FL schemes with homogeneous model settings.

FedMP: Federated Learning Through Adaptive Model Pruning in Heterogeneous Edge Computing

Computation and Communication Efficient Federated Learning With Adaptive Model Pruning

FedPSE: Personalized Sparsification with Element-wise Aggregation for Federated Learning

Efficient Federated Learning with Adaptive Channel Pruning for Edge Devices

FedPE: Adaptive Model Pruning-Expanding for Federated Learning on Mobile Devices

Adaptive Model Pruning for Communication and Computation Efficient Wireless Federated Learning

FedPrune: Personalized and Communication-Efficient Federated Learning on Non-IID Data

Accelerating Federated Learning with Data and Model Parallelism in Edge Computing

FedPAGE: Pruning Adaptively Toward Global Efficiency of Heterogeneous Federated Learning

FedEdge: Accelerating Edge-Assisted Federated Learning

Like Attracts Like: Personalized Federated Learning in Decentralized Edge Computing

Personalized Federated Learning Incorporating Adaptive Model Pruning at the Edge

Efficient federated learning on resource-constrained edge devices based on model pruning

Efficient Federated Learning Using Dynamic Update and Adaptive Pruning with Momentum on Shared Server Data

Adaptive Model Pruning and Personalization for Federated Learning Over Wireless Networks

NestFL: Enhancing Federated Learning Through Nested Multi-Capacity Model Pruning in Heterogeneous Edge Computing

On the Convergence of Heterogeneous Federated Learning with Arbitrary Adaptive Online Model Pruning

Model Pruning-enabled Federated Split Learning for Resource-constrained Devices in Artificial Intelligence Empowered Edge Computing Environment

Federated Learning With Client Selection and Gradient Compression in Heterogeneous Edge Systems

Straggler-resilient Federated Learning: Tackling Computation Heterogeneity with Layer-wise Partial Model Training in Mobile Edge Network

FedDUAP: Federated Learning with Dynamic Update and Adaptive Pruning Using Shared Data on the Server