Abstract:With the rapid expansion of edge devices, such as IoT devices, where crucial data needed for machine learning applications is generated, it becomes essential to promote their participation in privacy-preserving Federated Learning (FL) systems. The best way to achieve this desiderate is by reducing their training workload to match their constrained computational resources. While prior FL research has address the workload constrains by introducing lightweight models on the edge, limited attention has been given to optimizing on-device training efficiency through reducing the amount of data need during training. In this work, we propose FedFT-EDS, a novel approach that combines Fine-Tuning of partial client models with Entropy-based Data Selection to reduce training workloads on edge devices. By actively selecting the most informative local instances for learning, FedFT-EDS reduces training data significantly in FL and demonstrates that not all user data is equally beneficial for FL on all rounds. Our experiments on CIFAR-10 and CIFAR-100 show that FedFT-EDS uses only 50% user data while improving the global model performance compared to baseline methods, FedAvg and FedProx. Importantly, FedFT-EDS improves client learning efficiency by up to 3 times, using one third of training time on clients to achieve an equivalent performance to the baselines. This work highlights the importance of data selection in FL and presents a promising pathway to scalable and efficient Federate Learning.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: in Federated Learning (FL), how to improve the participation and efficiency of resource - constrained devices by reducing the training workload of edge devices. Specifically, the paper proposes a novel method FedFT - EDS (Federated Fine - Tuning with Entropy - based Data Selection), aiming to reduce the training burden of edge devices through partial model fine - tuning and an entropy - based data selection strategy. ### Core Problems of the Paper 1. **Reducing the Computational Burden of Edge Devices**: With the popularization of Internet of Things (IoT) devices, a large amount of critical data is generated on these resource - constrained edge devices. In order to enable these devices to participate in the privacy - protected Federated Learning system, their training workload must be reduced to match their limited computational resources. 2. **Optimizing Local Training Efficiency**: Although previous Federated Learning research has mainly focused on introducing lightweight models to reduce the computational burden, less attention has been paid to optimizing local training efficiency by reducing the amount of data required for training. ### Solutions The method FedFT - EDS proposed in the paper combines the following two strategies: - **Partial Model Fine - Tuning**: The client only needs to fine - tune a part of the model instead of the entire model, thereby significantly reducing the computational burden. - **Entropy - Based Data Selection**: By calculating the entropy value of each data sample, the most informative samples are selected for training, further reducing the amount of training data required. ### Main Contributions 1. **Effective Data Selection**: An entropy - based data selection method is introduced, and the hardened softmax activation function is used to efficiently reduce the computational burden of the client, especially in the non - IID (non - independent and identically distributed) setting. 2. **Fine - Tuning Strategy**: A pre - trained global model is used for fine - tuning to mitigate the impact of system and data heterogeneity and improve performance. 3. **Insight into Data Heterogeneity**: Experiments show that not all client data are of equal value to Federated Learning. By strategically selecting training data, performance can be significantly improved even when only using part of the data. ### Experimental Results The experimental results show that FedFT - EDS outperforms existing Federated Learning baseline methods (such as FedAvg and FedProx) on the CIFAR - 10 and CIFAR - 100 datasets. Specifically: - FedFT - EDS can improve the global model performance using only 50% of the user data and is three times faster than the baseline methods. - Under strong data heterogeneity conditions, the pre - trained model shows stronger robustness, resulting in an approximately 8% improvement in model performance. ### Summary This paper effectively reduces the training workload of edge devices and improves the efficiency and scalability of Federated Learning by combining partial model fine - tuning and entropy - based data selection. This provides a new way for resource - constrained devices to widely participate in Federated Learning.

Federated Learning with Workload Reduction through Partial Training of Client Models and Entropy-Based Data Selection

FedPSE: Personalized Sparsification with Element-wise Aggregation for Federated Learning

Client Selection for Federated Learning With Non-IID Data in Mobile Edge Computing

Data Selection for Efficient Model Update in Federated Learning

Enhancing Federated Learning Convergence with Dynamic Data Queue and Data Entropy-driven Participant Selection

Federated Learning Using Sparse-Adaptive Model Selection for Embedded Edge Computing

Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge

Efficient Data Distribution Estimation for Accelerated Federated Learning

A Cluster-Driven Adaptive Training Approach for Federated Learning

FedDA: Resource-adaptive Federated Learning with Dual-Alignment Aggregation Optimization for Heterogeneous Edge Devices

Toward efficient resource utilization at edge nodes in federated learning

Sample-level Data Selection for Federated Learning.

Contextual Client Selection for Efficient Federated Learning over Edge Devices

Learn More by Using Less: Distributed Learning with Energy-Constrained Devices

Towards addressing aggregation deviation for model training in resource-scarce edge environment

Towards Fairer and More Efficient Federated Learning via Multidimensional Personalized Edge Models

FedEP: Tailoring Attention to Heterogeneous Data Distribution with Entropy Pooling for Decentralized Federated Learning

Energy-efficient client selection in federated learning with heterogeneous data on edge

FedAda: Fast-convergent adaptive federated learning in heterogeneous mobile edge computing environment

Local Learning Matters: Rethinking Data Heterogeneity in Federated Learning

Federated Learning Optimization Algorithm for Automatic Weight Optimal