Abstract:The data heterogeneity across devices and the limited communication resources, e.g., bandwidth and energy, are two of the main bottlenecks for wireless federated learning (FL). To tackle these challenges, we first devise a novel FL framework with partial model aggregation (PMA). This approach aggregates the lower layers of neural networks, responsible for feature extraction, at the parameter server while keeping the upper layers, responsible for complex pattern recognition, at devices for personalization. The proposed PMA-FL is able to address the data heterogeneity and reduce the transmitted information in wireless channels. Then, we derive a convergence bound of the framework under a non-convex loss function setting to reveal the role of unbalanced data size in the learning performance. On this basis, we maximize the scheduled data size to minimize the global loss function through jointly optimize the device scheduling, bandwidth allocation, computation and communication time division policies with the assistance of Lyapunov optimization. Our analysis reveals that the optimal time division is achieved when the communication and computation parts of PMA-FL have the same power. We also develop a bisection method to solve the optimal bandwidth allocation policy and use the set expansion algorithm to address the device scheduling policy. Compared with the benchmark schemes, the proposed PMA-FL improves 3.13\% and 11.8\% accuracy on two typical datasets with heterogeneous data distribution settings, i.e., MINIST and CIFAR-10, respectively. In addition, the proposed joint dynamic device scheduling and resource management approach achieve slightly higher accuracy than the considered benchmarks, but they provide a satisfactory energy and time reduction: 29\% energy or 20\% time reduction on the MNIST; and 25\% energy or 12.5\% time reduction on the CIFAR-10.

Towards Efficient Federated Learning Framework Via Selective Aggregation of Models

SAM: an Efficient Approach with Selective Aggregation of Models in Federated Learning

FedPSE: Personalized Sparsification with Element-wise Aggregation for Federated Learning

Optimizing Federated Learning on Non-IID Data Using Local Shapley Value.

Enhancing Edge-Assisted Federated Learning with Asynchronous Aggregation and Cluster Pairing

A Communication-Efficient Hierarchical Federated Learning Framework via Shaping Data Distribution at Edge

Robust Model Aggregation for Heterogeneous Federated Learning: Analysis and Optimizations

Communication-Efficient Model Aggregation with Layer Divergence Feedback in Federated Learning

Efficient and Less Centralized Federated Learning

Communication Efficient Federated Learning With Heterogeneous Structured Client Models

Federated Select: A Primitive for Communication- and Memory-Efficient Federated Learning

Efficient Wireless Federated Learning with Partial Model Aggregation

Federated Learning Using Sparse-Adaptive Model Selection for Embedded Edge Computing

Federated learning with workload-aware client scheduling in heterogeneous systems

SHARE: Shaping Data Distribution at Edge for Communication-Efficient Hierarchical Federated Learning

An Efficient Asynchronous Federated Learning Protocol for Edge Devices

Latency Aware Semi-synchronous Client Selection and Model Aggregation for Wireless Federated Learning

Computation and Communication Efficient Federated Learning With Adaptive Model Pruning

Efficient Model Compression for Hierarchical Federated Learning

Efficient Data Distribution Estimation for Accelerated Federated Learning

Is Aggregation the Only Choice? Federated Learning via Layer-wise Model Recombination