Abstract:This article studies the unmanned aerial vehicle (UAV)-assisted wireless powered network, where a UAV is dispatched to wirelessly charge multiple ground nodes (GNs) by using radio frequency (RF) energy transfer and then the GNs use their harvested energy to upload the sensed information to the UAV. At each moment, the UAV is scheduled to charge the GNs or only one GN is scheduled to upload its data. An optimization problem is formulated to minimize the average Age of Information (AoI) of the GNs by jointly optimizing the trajectory of the UAV and the scheduling of information transmission and energy harvesting of GNs. As the problem is a combinational optimization problem with a set of binary variables, it is difficult to be solved. Thus, it is modeled as a Markov problem with large state spaces and a deep <span class="mjpage"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="1.838ex" height="2.509ex" style="vertical-align: -0.671ex;" viewBox="0 -791.3 791.5 1080.4" role="img" focusable="false" xmlns="http://www.w3.org/2000/svg"><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"> <use xlink:href="#MJMATHI-51" x="0" y="0"></use></g></svg></span> network (DQN)-based scheme is proposed to find its near-optimal solution on the basis of the deep reinforcement learning (DRL) framework. Two nets are structured with artificial neural network (ANN), where one is for evaluating the reward of the action performed in current state, and the other is for predicting realistic action. The corresponding state spaces, the efficient action spaces, and reward function are designed. Simulation results demonstrate the convergence of the proposed DQN scheme, which also show that the proposed DQN scheme gets much smaller average AoI than the three other known schemes. Moreover, by involving the energy punishment in the reward, the UAV may save its energy but yield higher AoI. Additionally, the effects of the packet size, the transmit power, and the distribution area of GNs on the GNs' average AoI are also discussed, which are expected to provide some useful insights.<svg xmlns="http://www.w3.org/2000/svg" style="display: none;"><defs id="MathJax_SVG_glyphs"><path stroke-width="1" id="MJMATHI-51" d="M399 -80Q399 -47 400 -30T402 -11V-7L387 -11Q341 -22 303 -22Q208 -22 138 35T51 201Q50 209 50 244Q50 346 98 438T227 601Q351 704 476 704Q514 704 524 703Q621 689 680 617T740 435Q740 255 592 107Q529 47 461 16L444 8V3Q444 2 449 -24T470 -66T516 -82Q551 -82 583 -60T625 -3Q631 11 638 11Q647 11 649 2Q649 -6 639 -34T611 -100T557 -165T481 -194Q399 -194 399 -87V-80ZM636 468Q636 523 621 564T580 625T530 655T477 665Q429 665 379 640Q277 591 215 464T153 216Q153 110 207 59Q231 38 236 38V46Q236 86 269 120T347 155Q372 155 390 144T417 114T429 82T435 55L448 64Q512 108 557 185T619 334T636 468ZM314 18Q362 18 404 39L403 49Q399 104 366 115Q354 117 347 117Q344 117 341 117T337 118Q317 118 296 98T274 52Q274 18 314 18Z"></path></defs></svg>

Learning-Based Energy Minimization Optimization for IRS-Assisted Master-Auxiliary-UAV-Enabled Wireless-Powered IoT Networks

Optimization for Master-UAV-powered Auxiliary-Aerial-IRS-assisted IoT Networks: An Option-based Multi-agent Hierarchical Deep Reinforcement Learning Approach

Energy Efficiency Optimization of IRS and UAV-Assisted Wireless Powered Edge Networks

Penalized Reinforcement Learning-Based Energy-Efficient UAV-RIS Assisted Maritime Uplink Communications Against Jamming

Energy-Efficient Data Collection and Wireless Power Transfer Using A MIMO Full-Duplex UAV

Joint Power and Trajectory Optimization for IRS-aided Master-Auxiliary-UAV-powered IoT Networks

Energy Minimization for IRS-assisted UAV-empowered Wireless Communications

Multi-UAV Path Learning for Age and Power Optimization in IoT With UAV Battery Recharge

UAV-Aided Lifelong Learning for AoI and Energy Optimization in Non-Stationary IoT Networks

Joint Optimization of Data Acquisition and Trajectory Planning for UAV-Assisted Wireless Powered Internet of Things

Energy Efficiency Maximization for Multi-UAV-IRS-Assisted Marine Vehicle Systems

Fair Energy-Efficient Resource Optimization for Multi-UAV Enabled Internet of Things

Multi-Task Transfer Deep Reinforcement Learning for Timely Data Collection in Rechargeable-UAV-aided IoT Networks

Reinforcement-Learning-Based Optimization on Energy Efficiency in UAV Networks for IoT

Joint Optimization of Charging Station Placement and UAV Trajectory for Fresh Data Collection

Multi-Objective Optimization for UAV-Assisted Wireless Powered IoT Networks Based on Extended DDPG Algorithm

Deep Reinforcement Learning for Energy-Efficient Fresh Data Collection in Rechargeable UAV-assisted IoT Networks

Multitask Transfer Deep Reinforcement Learning for Timely Data Collection in Rechargeable-UAV-Aided IoT Networks

Rechargeable Multi-UAV Aided Seamless Coverage for QoS-Guaranteed IoT Networks

Age of Information Minimization for UAV-assisted Internet of Things Networks: A Safe Actor-Critic with Policy Distillation Approach

Average AoI Minimization in UAV-Assisted Data Collection With RF Wireless Power Transfer: A Deep Reinforcement Learning Scheme