Abstract:The conventional model aggregation-based federated learning (FL) approach requires all local models to have the same architecture, which fails to support practical scenarios with heterogeneous local models. Moreover, frequent model exchange is costly for resource-limited wireless networks since modern deep neural networks usually have over a million parameters. To tackle these challenges, we propose a novel knowledge-aided FL (KFL) framework, which aggregates light high-level data features, namely knowledge, in the per-round learning process. This framework allows devices to design their machine-learning models independently and reduces the communication overhead in the training process. We then theoretically analyze the convergence bound of the proposed framework, revealing that scheduling more data volume in each round helps to improve the learning performance. In addition, large data volume should be scheduled in early rounds if the total scheduled data volume during the entire learning course is fixed. Inspired by this, we define a new objective function, i.e., the weighted scheduled data sample volume, to transform the inexplicit global loss minimization problem into a tractable one for device scheduling, bandwidth allocation, and power control. To deal with unknown time-varying wireless channels, we transform the considered problem into a deterministic problem for each round with the assistance of the Lyapunov optimization framework. Then, we derive the optimal bandwidth allocation and power control solution and develop an efficient online device scheduling algorithm to achieve an energy-learning trade-off in the learning process. Experimental results on MNIST and CIFAR-10 show that the proposed KFL is capable of reducing over 99% communication overhead while achieving better learning performance than the conventional model aggregation-based algorithms.

Bandwidth Allocation for Federated Learning with Wireless Providers and Cost Constraints

Resource Allocation for Multi-Task Federated Learning Algorithm over Wireless Communication Networks

Bandwidth Allocation for Multiple Federated Learning Services in Wireless Edge Networks

Message Passing Based Wireless Federated Learning Via Analog Message Aggregation

Client Selection and Bandwidth Allocation for Federated Learning: An Online Optimization Perspective

Client Selection and Bandwidth Allocation in Wireless Federated Learning Networks: A Long-Term Perspective

Joint Device Scheduling and Resource Allocation for Latency Constrained Wireless Federated Learning

Efficiency-Boosting Federated Learning in Wireless Networks: A Long-Term Perspective

Joint User Association and Resource Allocation for Wireless Hierarchical Federated Learning with IID and Non-IID Data

A Joint Learning and Communications Framework for Federated Learning over Wireless Networks

Federated Learning Over Wireless Channels: Dynamic Resource Allocation and Task Scheduling

Device Scheduling with Fast Convergence for Wireless Federated Learning

Delay Minimization for Federated Learning Over Wireless Communication Networks

Joint Client Selection and Bandwidth Allocation of Wireless Federated Learning by Deep Reinforcement Learning

Knowledge-aided Federated Learning for Energy-limited Wireless Networks

Multi-Objective Optimization for Bandwidth-Limited Federated Learning in Wireless Edge Systems

System Optimization of Federated Learning Networks with A Constrained Latency

Federated Learning over Wireless Networks: Convergence Analysis and Resource Allocation

Federated Learning over Wireless IoT Networks with Optimized Communication and Resources

Energy Efficient Federated Learning Over Wireless Communication Networks

Adaptive User Selection and Bandwidth Allocation for Fast Convergence of Federated Learning in Wireless Networks