Abstract:Most existing wireless federated learning (FL) studies focused on homogeneous model settings where devices train identical local models. In this setting, the devices with poor communication and computation capabilities may delay the global model update and degrade the performance of FL. Moreover, in the homogenous model settings, the scale of the global model is restricted by the device with the lowest capability. To tackle these challenges, this work proposes an adaptive model pruning-based FL (AMP-FL) framework, where the edge server dynamically generates sub-models by pruning the global model for devices’ local training to adapt their heterogeneous computation capabilities and time-varying channel conditions. Since the involvement of diverse structures of devices’ sub-models in the global model updating may negatively affect the training convergence, we propose compensating for the gradients of pruned model regions by devices’ historical gradients. We then introduce an age of information (AoI) metric to characterize the staleness of local gradients and theoretically analyze the convergence behaviour of AMP-FL. The convergence bound suggests scheduling devices with large AoI of gradients and pruning the model regions with small AoI for devices to improve the learning performance. Inspired by this, we define a new objective function, i.e., the average AoI of local gradients, to transform the inexplicit global loss minimization problem into a tractable one for device scheduling, model pruning, and resource block (RB) allocation design. Through detailed analysis, we derive the optimal model pruning strategy and transform the RB allocation problem into equivalent linear programming that can be effectively solved. Experimental results demonstrate the effectiveness and superiority of the proposed approaches. The proposed AMP-FL is capable of achieving 1.9x and 1.6x speed up for FL on MNIST and CIFAR-10 datasets in comparison with the FL schemes with homogeneous model settings.

AdaptiveFL: Adaptive Heterogeneous Federated Learning for Resource-Constrained AIoT Systems

AdaptFL: Adaptive Federated Learning Framework for Heterogeneous Devices

FlexFL: Heterogeneous Federated Learning via APoZ-Guided Flexible Pruning in Uncertain Scenarios

Efficient Adaptive Federated Learning in Resource-Constrained IoT Environments

AdapterFL: Adaptive Heterogeneous Federated Learning for Resource-constrained Mobile Computing Systems

Efficient Adaptive Federated Optimization of Federated Learning for IoT

Adaptive Federated Learning for Resource-Constrained IoT Devices Through Edge Intelligence and Multi-Edge Clustering

FedASA: A Personalized Federated Learning with Adaptive Model Aggregation for Heterogeneous Mobile Edge Computing

Adaptive Federated Learning on Non-IID Data with Resource Constraint

PervasiveFL: Pervasive Federated Learning for Heterogeneous IoT Systems.

Efficient Federated Learning for Cloud-Based AIoT Applications

Computation and Communication Efficient Adaptive Federated Optimization of Federated Learning for Internet of Things

Hybrid IoT Device Selection with Knowledge Transfer for Federated Learning

Adaptive Model Pruning for Communication and Computation Efficient Wireless Federated Learning

Efficient Federated Learning for AIoT Applications Using Knowledge Distillation

FedSAE: A Novel Self-Adaptive Federated Learning Framework in Heterogeneous Systems

DDPG-AdaptConfig: A Deep Reinforcement Learning Framework for Adaptive Device Selection and Training Configuration in Heterogeneity Federated Learning

Adaptive Model Pruning and Personalization for Federated Learning Over Wireless Networks

AEDFL: Efficient Asynchronous Decentralized Federated Learning with Heterogeneous Devices

Have Your Cake and Eat It Too: Toward Efficient and Accurate Split Federated Learning

Towards Fast and Accurate Federated Learning with Non-Iid Data for Cloud-Based IoT Applications