Abstract:Unmanned aerial vehicles (UAVs) have emerged as a promising candidate solution for data collection of large-scale wireless sensor networks (WSNs). In this paper, we investigate a UAV-aided WSN, where cluster heads (CHs) receive data from their member nodes, and a UAV is dispatched to collect data from CHs along the planned trajectory. We aim to minimize the total energy consumption of the UAV-WSN system in a complete round of data collection. Toward this end, we formulate the energy consumption minimization problem as a constrained combinatorial optimization problem by jointly selecting CHs from nodes within clusters and planning the UAV's visiting order to the selected CHs. The formulated energy consumption minimization problem is NP-hard, and hence, hard to solve optimally. In order to tackle this challenge, we propose a novel deep reinforcement learning (DRL) technique, pointer network-A* (Ptr-A*), which can efficiently learn from experiences the UAV trajectory policy for minimizing the energy consumption. The UAV's start point and the WSN with a set of pre-determined clusters are fed into the Ptr-A*, and the Ptr-A* outputs a group of CHs and the visiting order to these CHs, i.e., the UAV's trajectory. The parameters of the Ptr-A* are trained on small-scale clusters problem instances for faster training by using the actor-critic algorithm in an unsupervised manner. At inference, three search strategies are also proposed to improve the quality of solutions. Simulation results show that the trained models based on 20-clusters and 40-clusters have a good generalization ability to solve the UAV's trajectory planning problem in WSNs with different numbers of clusters, without the need to retrain the models. Furthermore, the results show that our proposed DRL algorithm outperforms two baseline techniques.

Enhancing Sustainable Data Collection with Unmanned Aerial Vehicles: A Deep Reinforcement Learning Approach

Energy-Efficient UAV-Enabled Data Collection via Wireless Charging: A Reinforcement Learning Approach

Deep Reinforcement Learning for Energy-Efficient Fresh Data Collection in Rechargeable UAV-assisted IoT Networks

UAV Trajectory Optimization for Large-Scale and Low-Power Data Collection: An Attention-Reinforced Learning Scheme

A Deep Reinforcement Learning Approach for Multi-UAV-Assisted Data Collection in Wireless Powered IoT Networks

Research on the UAV-aided Data Collection and Trajectory Design Based on the Deep Reinforcement Learning

An Efficient Reinforcement Learning Game Framework for UAV-Enabled Wireless Sensor Network Data Collection

UAV Trajectory Planning in Wireless Sensor Networks for Energy Consumption Minimization by Deep Reinforcement Learning

A Deep Reinforcement Learning Approach to Energy-harvesting UAV-aided Data Collection

Deep Reinforcement Learning Based Data Collection with Charging Stations

Multi-Task Transfer Deep Reinforcement Learning for Timely Data Collection in Rechargeable-UAV-aided IoT Networks

Deep Reinforcement Learning for Joint Trajectory Planning, Transmission Scheduling, and Access Control in UAV-Assisted Wireless Sensor Networks

Deep Reinforcement Learning for Fresh Data Collection in UAV-assisted IoT Networks

Joint Optimization of Multi-UAV-Assisted Data Collection and Energy Replenishment Via Transfer Learning Aided Deep Reinforcement Learning

DRL-UTPS: DRL-Based Trajectory Planning for Unmanned Aerial Vehicles for Data Collection in Dynamic IoT Network

Energy Efficient UAV-Aassisted IoT Data Collection: A Graph-Based Deep Reinforcement Learning Approach

UAV Trajectory Planning for Data Collection in Internet of Vehicles: A Reinforcement Learning Approach

Meta-Reinforcement Learning for Timely and Energy-efficient Data Collection in Solar-powered UAV-assisted IoT Networks

UAV-Enabled Diverse Data Collection Via Integrated Sensing and Communication Functions Based on Deep Reinforcement Learning

Multitask Transfer Deep Reinforcement Learning for Timely Data Collection in Rechargeable-UAV-Aided IoT Networks

Energy Consumption and Communication Quality Tradeoff for Logistics UAVs: A Hybrid Deep Reinforcement Learning Approach