Abstract:Unmanned aerial vehicles (UAVs) have emerged as a promising candidate solution for data collection of large-scale wireless sensor networks (WSNs). In this paper, we investigate a UAV-aided WSN, where cluster heads (CHs) receive data from their member nodes, and a UAV is dispatched to collect data from CHs along the planned trajectory. We aim to minimize the total energy consumption of the UAV-WSN system in a complete round of data collection. Toward this end, we formulate the energy consumption minimization problem as a constrained combinatorial optimization problem by jointly selecting CHs from nodes within clusters and planning the UAV's visiting order to the selected CHs. The formulated energy consumption minimization problem is NP-hard, and hence, hard to solve optimally. In order to tackle this challenge, we propose a novel deep reinforcement learning (DRL) technique, pointer network-A* (Ptr-A*), which can efficiently learn from experiences the UAV trajectory policy for minimizing the energy consumption. The UAV's start point and the WSN with a set of pre-determined clusters are fed into the Ptr-A*, and the Ptr-A* outputs a group of CHs and the visiting order to these CHs, i.e., the UAV's trajectory. The parameters of the Ptr-A* are trained on small-scale clusters problem instances for faster training by using the actor-critic algorithm in an unsupervised manner. At inference, three search strategies are also proposed to improve the quality of solutions. Simulation results show that the trained models based on 20-clusters and 40-clusters have a good generalization ability to solve the UAV's trajectory planning problem in WSNs with different numbers of clusters, without the need to retrain the models. Furthermore, the results show that our proposed DRL algorithm outperforms two baseline techniques.

Deep Reinforcement Learning-Based UAV Path Planning for Energy-Efficient Multitier Cooperative Computing in Wireless Sensor Networks

Deep Reinforcement Learning for Joint Trajectory Planning, Transmission Scheduling, and Access Control in UAV-Assisted Wireless Sensor Networks

UAV Trajectory Planning in Wireless Sensor Networks for Energy Consumption Minimization by Deep Reinforcement Learning

A Load-Balanced and Energy-Efficient Navigation Scheme for UAV-Mounted Mobile Edge Computing

Double Deep Q-learning Network-Based Path Planning in UAV-assisted Wireless Powered NOMA Communication Networks

An Efficient Reinforcement Learning Game Framework for UAV-Enabled Wireless Sensor Network Data Collection

Deep Reinforcement Learning based Path Planning for UAV-assisted Edge Computing Networks

Energy-Efficient Scheduling in UAV-Assisted Hierarchical Wireless Sensor Networks

Energy-aware Path Planning for Obtaining Fresh Updates in UAV-IoT MEC systems

Learning-Based UAV Path Planning for Data Collection with Integrated Collision Avoidance

Multi-UAV Assisted On-Demand Charging in Dense Wireless Rechargeable Sensor Networks

UAV Dispatch Planning for a Wireless Rechargeable Sensor Network for Bridge Monitoring

Multi-UAV WRSN charging path planning based on improved heed and IA-DRL

UAV Autonomous Navigation for Wireless Powered Data Collection with Onboard Deep Q-Network

A Deep Reinforcement Learning Approach for Multi-UAV-Assisted Data Collection in Wireless Powered IoT Networks

Deep Reinforcement Learning for Energy-Efficient Fresh Data Collection in Rechargeable UAV-assisted IoT Networks

DRL-Based Energy-Efficient Trajectory Planning, Computation Offloading, and Charging Scheduling in UAV-MEC Network.

On Designing Multi-UAV aided Wireless Powered Dynamic Communication via Hierarchical Deep Reinforcement Learning

Large-scale Path Planning and Time Window Allocation in UAV-assisted Wireless Sensor Networks with Variational Multi-dimensional Optimization

Hybrid energy‐Efficient distributed aided frog leaping dynamic A* with reinforcement learning for enhanced trajectory planning in UAV swarms large‐scale networks

Optimization Algorithms for UAV-and-MUV Cooperative Data Collection in Wireless Sensor Networks