Abstract:Unmanned aerial vehicle (UAV) has been widely deployed in efficient data collection for Internet of Things (IoT) networks. Information freshness in data collection can be characterized by the Age of Information (AoI). It is highly challenging to schedule multiple energy-constrained UAVs to improve information freshness especially when the generation instants of sensing samples are unpredictable. To deal with this issue, we leverage state-of-art reinforcement learning (RL) methods to design flight trajectories of UAVs without knowing the sampling mode each sensor node (SN) adopts. Each SN can sample the environment at periodical or random intervals. Multiple energy-constrained UAVs are dispatched to collect update packets from the SNs when flying over them. The UAV trajectory planning problem for AoI minimization is formulated as a Markov decision process (MDP). The objective is to minimize the average AoI of the SNs under the constraints of energy capacity and collision avoidance for the UAVs. Then, we propose two learning algorithms based on the Sarsa and value-decomposition network (VDN), respectively, which allow the UAVs to fulfill data collection tasks requested by the SNs. By learning directly from the environment, the Sarsa-based algorithm can approach the optimal policy asymptotically when certain conditions are satisfied. As one of the most popular multiagent deep RL methods, the VDN-based algorithm enables each UAV to make its own decision independently on its flight and data collection based on the partially observed network information. Simulation results validate the effectiveness of the proposed two learning-based algorithms compared with baseline policies.

Joint Power Control and UAV Trajectory Design for Information Freshness Via Deep Reinforcement Learning

AoI optimal UAV trajectory planning: A Deep Recurrent Reinforcement Learning Approach

Joint AoI-Aware UAVs Trajectory Planning and Data Collection in UAV-Based IoT Systems: A Deep Reinforcement Learning Approach

Deep RL-based Trajectory Planning for AoI Minimization in UAV-assisted IoT

Trajectory Design for Multi-UAV-Assisted Data Collection: A Multi-agent Deep Reinforcement Learning Approach

Multi-UAV Trajectory Design and Power Control Based on Deep Reinforcement Learning.

Deep Reinforcement Learning Approach for Joint Trajectory Design in Multi-UAV IoT Networks

Work Together to Keep Fresh: Hierarchical Learning for UAVs-assisted Data Time-Sensitive IoT

Joint Trajectory and Power Optimization in Multi-Type UAVs Network with Mean Field Q-Learning.

Trajectory Design for UAV-Based Internet of Things Data Collection: A Deep Reinforcement Learning Approach

AoI-driven Fresh Situation Awareness by UAV Swarm: Collaborative DRL-based Energy-Efficient Trajectory Control and Data Processing.

Deep Reinforcement Learning for Fresh Data Collection in UAV-assisted IoT Networks

Joint Trajectory and Scheduling Optimization for Age of Synchronization Minimization in UAV-Assisted Networks with Random Updates

AoI Optimal Trajectory Planning for Cooperative UAVs: A Multi-Agent Deep Reinforcement Learning Approach

Joint Trajectory and Power Optimization for Energy Efficient UAV Communication Using Deep Reinforcement Learning

Joint Cluster Head Selection and Trajectory Planning in UAV-Aided IoT Networks by Reinforcement Learning with Sequential Model

Decentralized Trajectory and Power Control Based on Multi-Agent Deep Reinforcement Learning in UAV Networks

Energy-AOI-Aware UAV-Assisted Data Collection: A Multi-Agent Deep Reinforcement Learning-Based Trajectory Optimization

Trajectory Design and Resource Allocation for Multi-UAV Networks: Deep Reinforcement Learning Approaches

Learning-Based Data Gathering for Information Freshness in UAV-Assisted IoT Networks

Three-dimensional deep reinforcement learning for trajectory and resource optimization in UAV communication systems