Abstract:Federated learning (FL) has been considered as a promising paradigm for enabling distributed machine learning (ML) in wireless networks. To address the limited energy capacity of wireless devices, we propose a simultaneous wireless information and power transfer (SWIPT) aided FL, in which one FL server (FLS) co-located at a cellular base station (BS) uses SWIPT to simultaneously broadcast the global model to wireless user-devices (UDs) and provide wireless power transfer to them. The UDs then use the harvested energy to train their local models and further transmit the local models to the FLS for aggregation. To improve the spectrum efficiency, we consider that the UDs form a non-orthogonal multiple access (NOMA) group for simultaneously sending their local models over the same spectrum channel. Taking the UDs' time-varying available energy and channel conditions into account, we propose a dynamic optimization of the UDs-scheduling, the BS's transmit-power allocation, and the UDs' power-splitting factors for SWIPT, with the objective of minimizing the long-term energy consumption while ensuring the FL convergence. The optimization problem, however, is challenging to solve since it is a finite-horizon dynamic programming problem but with an unknown stopping time, and moreover, the action space covers both discrete and continuous variables. To address these difficulties, we first execute a series of equivalent transformations to reduce the number of decision variables and then formulate the problem as a stochastic shortest path problem, based on which we propose an actor-critic deep reinforcement learning algorithm with the proximal policy optimization to efficiently learn the policy that dynamically adjusts the UDs-scheduling for FL as well as the BS's transmit-power for SWIPT. Numerical results validate the effectiveness and performance of our proposed algorithm. The results demonstrate that our proposed algorithm can effectively reduce the long-term energy consumption in comparison with two baseline algorithms.

Joint Device Participation, Dataset Management, and Resource Allocation in Wireless Federated Learning Via Deep Reinforcement Learning

Joint Client Selection and Bandwidth Allocation of Wireless Federated Learning by Deep Reinforcement Learning

Deep Reinforcement Learning for Multi-Functional RIS-Aided Over-the-Air Federated Learning in Internet of Robotic Things

Deep Reinforcement Learning-Empowered Federated Learning for Wireless Clients with Energy and Bandwidth Constraints

Collaborative Optimization of Wireless Communication and Computing Resource Allocation based on Multi-Agent Federated Weighting Deep Reinforcement Learning

Deep Reinforcement Learning for Energy Efficiency Maximization in SWIPT-Based Over-the-Air Federated Learning

Deep Reinforcement Learning for Over-the-Air Federated Learning in SWIPT-Enabled IoT Networks

Privacy-Preserving Resource Allocation for Asynchronous Federated Learning

Federated Multi-Agent Deep Reinforcement Learning for Resource Allocation of Vehicle-to-Vehicle Communications

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

Federated Learning Over Wireless Channels: Dynamic Resource Allocation and Task Scheduling

Exploring Deep Reinforcement Learning-Assisted Federated Learning for Online Resource Allocation in Privacy-Persevering EdgeIoT

Dynamic Resource Management for Federated Edge Learning With Imperfect CSI: A Deep Reinforcement Learning Approach

An Optimization Method for Non-IID Federated Learning Based on Deep Reinforcement Learning

Asynchronous Multi-Model Dynamic Federated Learning over Wireless Networks: Theory, Modeling, and Optimization

On-Demand Model and Client Deployment in Federated Learning with Deep Reinforcement Learning

Towards Dynamic Resource Allocation and Client Scheduling in Hierarchical Federated Learning: A Two-Phase Deep Reinforcement Learning Approach

Joint Device Scheduling and Resource Allocation for Latency Constrained Wireless Federated Learning

Dynamic User-Scheduling and Power Allocation for SWIPT Aided Federated Learning: A Deep Learning Approach

Dap-FL: Federated Learning Flourishes by Adaptive Tuning and Secure Aggregation

Computation Offloading and Resource Allocation in F-RANs: A Federated Deep Reinforcement Learning Approach