Abstract:Federated learning (FL) enables edge nodes to collaboratively contribute to constructing a global model without sharing their data. This is accomplished by devices computing local, private model updates that are then aggregated by a server. However, computational resource constraints and network communication can become a severe bottleneck for larger model sizes typical for deep learning applications. Edge nodes tend to have limited hardware resources (RAM, CPU), and the network bandwidth and reliability at the edge is a concern for scaling federated fleet applications. In this paper, we propose and evaluate a FL strategy inspired by transfer learning in order to reduce resource utilization on devices, as well as the load on the server and network in each global training round. For each local model update, we randomly select layers to train, freezing the remaining part of the model. In doing so, we can reduce both server load and communication costs per round by excluding all untrained layer weights from being transferred to the server. The goal of this study is to empirically explore the potential trade-off between resource utilization on devices and global model convergence under the proposed strategy. We implement the approach using the federated learning framework FEDn. A number of experiments were carried out over different datasets (CIFAR-10, CASA, and IMDB), performing different tasks using different deep-learning model architectures. Our results show that training the model partially can accelerate the training process, efficiently utilizes resources on-device, and reduce the data transmission by around 75% and 53% when we train 25%, and 50% of the model layers, respectively, without harming the resulting global model accuracy.

Federated Learning with Workload Reduction through Partial Training of Client Models and Entropy-Based Data Selection

Client Selection for Federated Learning With Non-IID Data in Mobile Edge Computing

Data Selection for Efficient Model Update in Federated Learning

Enhancing Federated Learning Convergence with Dynamic Data Queue and Data Entropy-driven Participant Selection

Federated Learning Using Sparse-Adaptive Model Selection for Embedded Edge Computing

Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge

Efficient Data Distribution Estimation for Accelerated Federated Learning

A Cluster-Driven Adaptive Training Approach for Federated Learning

FedDA: Resource-adaptive Federated Learning with Dual-Alignment Aggregation Optimization for Heterogeneous Edge Devices

Toward efficient resource utilization at edge nodes in federated learning

Sample-level Data Selection for Federated Learning.

Contextual Client Selection for Efficient Federated Learning over Edge Devices

Importance-aware Data Selection and Resource Allocation for Hierarchical Federated Edge Learning

FedProf: Selective Federated Learning Based on Distributional Representation Profiling

Learn More by Using Less: Distributed Learning with Energy-Constrained Devices

FedEdge: Accelerating Edge-Assisted Federated Learning

Towards addressing aggregation deviation for model training in resource-scarce edge environment

Towards Fairer and More Efficient Federated Learning via Multidimensional Personalized Edge Models

FedEP: Tailoring Attention to Heterogeneous Data Distribution with Entropy Pooling for Decentralized Federated Learning

Energy-efficient client selection in federated learning with heterogeneous data on edge

Federated Learning Client Selection Mechanism under System and Data Heterogeneity