Abstract:Federated Learning (FL) is a decentralized approach for collaborative model training on edge devices. This distributed method of model training offers advantages in privacy, security, regulatory compliance, and cost-efficiency. Our emphasis in this research lies in addressing statistical complexity in FL, especially when the data stored locally across devices is not identically and independently distributed (non-IID). We have observed an accuracy reduction of up to approximately 10\% to 30\%, particularly in skewed scenarios where each edge device trains with only 1 class of data. This reduction is attributed to weight divergence, quantified using the Euclidean distance between device-level class distributions and the population distribution, resulting in a bias term (\(\delta_k\)). As a solution, we present a method to improve convergence in FL by creating a global subset of data on the server and dynamically distributing it across devices using a Dynamic Data queue-driven Federated Learning (DDFL). Next, we leverage Data Entropy metrics to observe the process during each training round and enable reasonable device selection for aggregation. Furthermore, we provide a convergence analysis of our proposed DDFL to justify their viability in practical FL scenarios, aiming for better device selection, a non-sub-optimal global model, and faster convergence. We observe that our approach results in a substantial accuracy boost of approximately 5\% for the MNIST dataset, around 18\% for CIFAR-10, and 20\% for CIFAR-100 with a 10\% global subset of data, outperforming the state-of-the-art (SOTA) aggregation algorithms.

Federated Learning Optimization: A Comparative Study of Data and Model Exchange Strategies in Dynamic Networks

Federated Learning over Wireless Networks: Optimization Model Design and Analysis

Optimizing Federated Learning on Non-IID Data Using Local Shapley Value.

Exploring System-Heterogeneous Federated Learning with Dynamic Model Selection

Data Selection for Efficient Model Update in Federated Learning

A Field Guide to Federated Optimization

Towards Efficient Federated Learning Framework Via Selective Aggregation of Models

Multi-objective Evolutionary Federated Learning

Decentralized Federated Learning with Intermediate Results in Mobile Edge Computing

FedExg: Federated Learning with Model Exchange

Selective Federated Learning for Mobile Edge Intelligence

FedHe: Heterogeneous Models and Communication-Efficient Federated Learning

A Survey of What to Share in Federated Learning: Perspectives on Model Utility, Privacy Leakage, and Communication Efficiency

Federated Optimization in Heterogeneous Networks

Federated Learning: Strategies for Improving Communication Efficiency

Efficient and Less Centralized Federated Learning

Asynchronous Multi-Model Dynamic Federated Learning over Wireless Networks: Theory, Modeling, and Optimization

Enhancing Federated Learning Convergence with Dynamic Data Queue and Data Entropy-driven Participant Selection

When Decentralized Optimization Meets Federated Learning

FedECADO: A Dynamical System Model of Federated Learning

Federated Learning Model Aggregation in Heterogenous Aerial and Space Networks