Abstract:Federated learning (FL) is an emerging distributed learning paradigm that allows multiple clients to collaborate on training a global model without sharing their local data. However, in practical heterogeneous edge device scenarios, FL faces the challenges of system heterogeneity and data heterogeneity, which leads to unfair participation and degraded global model performance. In this paper, we introduce FedDA, a resource- adaptive FL framework, which adapts to the client's computing resources by assigning heterogeneous models of different sizes. To improve the performance of heterogeneous model aggregation and adjust to non- independent and identically distributed (non-i.i.d.) data, we propose a dual-alignment aggregation optimization method, i.e., parameter feature space alignment and output space alignment. Specifically, FedDA exploits the permutation symmetry property of weight space to permutate the model parameter positions via an adaptive layer-wise matching method, thereby aligning models with significant deviations in parameter feature space. FedDA mitigates the imbalance in parameter quantity between smaller and larger models through parameter expansion. Additionally, FedDA maps client labels into a uniform embedding space through output space alignment, thus reducing model parameter deviations due to non-i.i.d. data without additional client-side computing overhead. We evaluate the performance of FedDA on benchmark datasets, including FashionMNIST, CIFAR10, CIFAR100 and AGNews. Experimental results demonstrate that FedDA achieves up to 8.71% improvement in model accuracy compared to baseline methods, highlighting its effectiveness in addressing the challenges of heterogeneity.

Accelerating Federated Learning with Adaptive Extra Local Updates Upon Edge Networks

FedDA: Faster Framework of Local Adaptive Gradient Methods via Restarted Dual Averaging

Delay-Aware Hierarchical Federated Learning

Efficient Federated Learning via Local Adaptive Amended Optimizer with Linear Speedup

AsyncFedED: Asynchronous Federated Learning with Euclidean Distance Based Adaptive Weight Aggregation

FedAgg: Adaptive Federated Learning with Aggregated Gradients

Achieving Linear Speedup in Asynchronous Federated Learning with Heterogeneous Clients

Accelerated Federated Learning with Decoupled Adaptive Optimization

Asynchronous Decentralized Federated Learning for Heterogeneous Devices

Adaptive Gradient Sparsification for Efficient Federated Learning: An Online Learning Approach

AEDFL: Efficient Asynchronous Decentralized Federated Learning with Heterogeneous Devices

Enhancing Edge-Assisted Federated Learning with Asynchronous Aggregation and Cluster Pairing

Towards addressing aggregation deviation for model training in resource-scarce edge environment

Enhancing Federated Learning Convergence with Dynamic Data Queue and Data Entropy-driven Participant Selection

DRAG: Divergence-based Adaptive Aggregation in Federated Learning on Non-IID Data

FedDA: Resource-adaptive Federated Learning with Dual-Alignment Aggregation Optimization for Heterogeneous Edge Devices

Asynchronous Online Federated Learning for Edge Devices with Non-IID Data

Adaptive Configuration for Heterogeneous Participants in Decentralized Federated Learning

Federated Learning with Unbiased Gradient Aggregation and Controllable Meta Updating

An Efficient Asynchronous Federated Learning Protocol for Edge Devices

Towards Efficient Asynchronous Federated Learning in Heterogeneous Edge Environments