Abstract:The conventional model aggregation-based federated learning (FL) approach requires all local models to have the same architecture, which fails to support practical scenarios with heterogeneous local models. Moreover, frequent model exchange is costly for resource-limited wireless networks since modern deep neural networks usually have over a million parameters. To tackle these challenges, we propose a novel knowledge-aided FL (KFL) framework, which aggregates light high-level data features, namely knowledge, in the per-round learning process. This framework allows devices to design their machine-learning models independently and reduces the communication overhead in the training process. We then theoretically analyze the convergence bound of the proposed framework, revealing that scheduling more data volume in each round helps to improve the learning performance. In addition, large data volume should be scheduled in early rounds if the total scheduled data volume during the entire learning course is fixed. Inspired by this, we define a new objective function, i.e., the weighted scheduled data sample volume, to transform the inexplicit global loss minimization problem into a tractable one for device scheduling, bandwidth allocation, and power control. To deal with unknown time-varying wireless channels, we transform the considered problem into a deterministic problem for each round with the assistance of the Lyapunov optimization framework. Then, we derive the optimal bandwidth allocation and power control solution and develop an efficient online device scheduling algorithm to achieve an energy-learning trade-off in the learning process. Experimental results on MNIST and CIFAR-10 show that the proposed KFL is capable of reducing over 99% communication overhead while achieving better learning performance than the conventional model aggregation-based algorithms.

Adaptive User Selection and Bandwidth Allocation for Fast Convergence of Federated Learning in Wireless Networks

Fast Convergence for Federated Learning in OFDMA Systems

Resource Allocation for Multi-Task Federated Learning Algorithm over Wireless Communication Networks

Adaptive Batchsize Selection and Gradient Compression for Wireless Federated Learning

Device Scheduling with Fast Convergence for Wireless Federated Learning

Joint User Selection and Resource Allocation for Fast Federated Edge Learning

Hierarchical Federated Learning with Adaptive Clustering on Non-IID Data

Federated Learning over Wireless Networks: Convergence Analysis and Resource Allocation

Joint User Association and Resource Allocation for Wireless Hierarchical Federated Learning with IID and Non-IID Data

Adaptive Modulation for Wireless Federated Learning

Asynchronous Federated Learning over Wireless Communication Networks.

A Joint Learning and Communications Framework for Federated Learning over Wireless Networks

Client Selection and Bandwidth Allocation for Federated Learning: An Online Optimization Perspective

Federated Learning over a Wireless Network: Distributed User Selection through Random Access

Convergence Analysis and System Design for Federated Learning over Wireless Networks.

Client Selection and Bandwidth Allocation in Wireless Federated Learning Networks: A Long-Term Perspective

Efficiency-Boosting Federated Learning in Wireless Networks: A Long-Term Perspective

Bandwidth Allocation for Federated Learning with Wireless Providers and Cost Constraints

Adaptive Heterogeneous Client Sampling for Federated Learning over Wireless Networks

Adaptive Federated Learning in Heterogeneous Wireless Networks with Independent Sampling

Knowledge-aided Federated Learning for Energy-limited Wireless Networks