Abstract:Maintaining the freshness of information in the Internet of Things (IoT) is a critical yet challenging problem. In this paper, we study cooperative data collection using multiple Unmanned Aerial Vehicles (UAVs) with the objective of minimizing the total average Age of Information (AoI). We consider various constraints of the UAVs, including kinematic, energy, trajectory, and collision avoidance, in order to optimize the data collection process. Specifically, each UAV, which has limited on-board energy, takes off from its initial location and flies over sensor nodes to collect update packets in cooperation with the other UAVs. The UAVs must land at their final destinations with non-negative residual energy after the specified time duration to ensure they have enough energy to complete their missions. It is crucial to design the trajectories of the UAVs and the transmission scheduling of the sensor nodes to enhance information freshness. We model the multi-UAV data collection problem as a Decentralized Partially Observable Markov Decision Process (Dec-POMDP), as each UAV is unaware of the dynamics of the environment and can only observe a part of the sensors. To address the challenges of this problem, we propose a multi-agent Deep Reinforcement Learning (DRL)-based algorithm with centralized learning and decentralized execution. In addition to the reward shaping, we use action masks to filter out invalid actions and ensure that the constraints are met. Simulation results demonstrate that the proposed algorithms can significantly reduce the total average AoI compared to the baseline algorithms, and the use of the action mask method can improve the convergence speed of the proposed algorithm.

Deep Reinforcement Learning for Efficient Data Collection in UAV-Aided Internet of Things

Deep Reinforcement Learning for Fresh Data Collection in UAV-assisted IoT Networks

A Deep Reinforcement Learning Approach for Multi-UAV-Assisted Data Collection in Wireless Powered IoT Networks

Deep Reinforcement Learning for Energy-Efficient Fresh Data Collection in Rechargeable UAV-assisted IoT Networks

Learning-Based Data Gathering for Information Freshness in UAV-Assisted IoT Networks

AoI-Energy-Aware UAV-Assisted Data Collection for IoT Networks: A Deep Reinforcement Learning Method

Timely Data Collection for UAV-based IoT networks: A Deep Reinforcement Learning Approach

A Deep Reinforcement Learning Approach to Energy-harvesting UAV-aided Data Collection

Meta-Reinforcement Learning for Timely and Energy-efficient Data Collection in Solar-powered UAV-assisted IoT Networks

UAV Resource Cooperation Based on Reinforcement Learning.

Deep Reinforcement Learning-Driven UAV Data Collection Path Planning: A Study on Minimizing AoI

Cooperative Data Collection with Multiple UAVs for Information Freshness in the Internet of Things

Joint Optimization of Multi-UAV-Assisted Data Collection and Energy Replenishment Via Transfer Learning Aided Deep Reinforcement Learning

UAV-Assisted Status Updates

Research on the UAV-aided Data Collection and Trajectory Design Based on the Deep Reinforcement Learning

Joint Speed Control and Energy Replenishment Optimization for UAV-assisted IoT Data Collection with Deep Reinforcement Transfer Learning

Joint AoI-Aware UAVs Trajectory Planning and Data Collection in UAV-Based IoT Systems: A Deep Reinforcement Learning Approach

Intelligent Reflecting Surface-Assisted Fresh Data Collection in UAV Communications

Intelligent joint trajectory design and resource allocation in UAV-based data harvesting system

Compound-Action DRL for Timely Data Collection in UAV-Assisted IoT with Stochastic Arrival