Abstract:The data heterogeneity across devices and the limited communication resources, e.g., bandwidth and energy, are two of the main bottlenecks for wireless federated learning (FL). To tackle these challenges, we first devise a novel FL framework with partial model aggregation (PMA). This approach aggregates the lower layers of neural networks, responsible for feature extraction, at the parameter server while keeping the upper layers, responsible for complex pattern recognition, at devices for personalization. The proposed PMA-FL is able to address the data heterogeneity and reduce the transmitted information in wireless channels. Then, we derive a convergence bound of the framework under a non-convex loss function setting to reveal the role of unbalanced data size in the learning performance. On this basis, we maximize the scheduled data size to minimize the global loss function through jointly optimize the device scheduling, bandwidth allocation, computation and communication time division policies with the assistance of Lyapunov optimization. Our analysis reveals that the optimal time division is achieved when the communication and computation parts of PMA-FL have the same power. We also develop a bisection method to solve the optimal bandwidth allocation policy and use the set expansion algorithm to address the device scheduling policy. Compared with the benchmark schemes, the proposed PMA-FL improves 3.13\% and 11.8\% accuracy on two typical datasets with heterogeneous data distribution settings, i.e., MINIST and CIFAR-10, respectively. In addition, the proposed joint dynamic device scheduling and resource management approach achieve slightly higher accuracy than the considered benchmarks, but they provide a satisfactory energy and time reduction: 29\% energy or 20\% time reduction on the MNIST; and 25\% energy or 12.5\% time reduction on the CIFAR-10.

Model-heterogeneous Federated Learning with Partial Model Training

Straggler-resilient Federated Learning: Tackling Computation Heterogeneity with Layer-wise Partial Model Training in Mobile Edge Network

FedDGP: Disentangling Global and Personal Models for Federated Learning

FedPSE: Personalized Sparsification with Element-wise Aggregation for Federated Learning

FedHe: Heterogeneous Models and Communication-Efficient Federated Learning

Efficient Wireless Federated Learning with Partial Model Aggregation

Computation and Communication Efficient Federated Learning With Adaptive Model Pruning

ChronusFed: Reinforcement-Based Adaptive Partial Training for Heterogeneous Federated Learning

FedLPS: Heterogeneous Federated Learning for Multiple Tasks with Local Parameter Sharing

Federated Model Heterogeneous Matryoshka Representation Learning

Completely Heterogeneous Federated Learning

Selective Knowledge Sharing for Personalized Federated Learning Under Capacity Heterogeneity

APFed: Attention-based Personalized Federated Learning in Model Heterogeneity Scenario.

Efficient Personalized Federated Learning via Sparse Model-Adaptation

FedLGA: Towards System-Heterogeneity of Federated Learning via Local Gradient Approximation

FedGH: Heterogeneous Federated Learning with Generalized Global Header

Towards Efficient Model-Heterogeneity Federated Learning for Large Models

Towards More Suitable Personalization in Federated Learning via Decentralized Partial Model Training

Federated Mutual Learning: a Collaborative Machine Learning Method for Heterogeneous Data, Models, and Objectives

Embracing Federated Learning: Enabling Weak Client Participation via Partial Model Training

FedPA: An adaptively partial model aggregation strategy in Federated Learning