Trajectory Planning for UAV-Assisted Data Collection in IoT Network: A Double Deep Q Network Approach

Shuqi Wang,Nan Qi,Hua Jiang,Ming Xiao,Haoxuan Liu,Luliang Jia,Dan Zhao

DOI: https://doi.org/10.3390/electronics13081592

IF: 2.9

2024-04-23

Electronics

Abstract:Unmanned aerial vehicles (UAVs) are becoming increasingly valuable as a new type of mobile communication device and autonomous decision-making device in many application areas, including the Internet of Things (IoT). UAVs have advantages over other stationary devices in terms of high flexibility. However, a UAV, as a mobile device, still faces some challenges in optimizing its trajectory for data collection. Firstly, the high complexity of the movement action and state space of the UAV's 3D trajectory is not negligible. Secondly, in unknown urban environments, a UAV must avoid obstacles accurately in order to ensure a safe flight. Furthermore, without a priori wireless channel characterization and ground device locations, a UAV must reliably and safely complete the data collection from the ground devices under the threat of unknown interference. All of these require the proposing of intelligent and automatic onboard trajectory optimization techniques. This paper transforms the trajectory optimization problem into a Markov decision process (MDP), and deep reinforcement learning (DRL) is applied to the data collection scenario. Specifically, the double deep Q-network (DDQN) algorithm is designed to address intelligent UAV trajectory planning that enables energy-efficient and safe data collection. Compared with the traditional algorithm, the DDQN algorithm is much better than the traditional Q-Learning algorithm, and the training time of the network is shorter than that of the deep Q-network (DQN) algorithm.

engineering, electrical & electronic,computer science, information systems,physics, applied

What problem does this paper attempt to address?

This paper discusses the trajectory planning problem of unmanned aerial vehicles (UAVs) in the Internet of Things (IoT) network for data collection. In the study, the authors transform the problem into a Markov decision process (MDP) and apply deep reinforcement learning (DRL), specifically the Double Deep Q-Network (DDQN) algorithm, to solve the energy-efficient and secure data collection trajectory planning for intelligent UAVs. The paper points out that traditional algorithms may not be suitable for complex environments, while DRL methods can learn strategies from trial and error without the need for specific environmental parameter modeling.

In practical scenarios, drones face challenges such as limited battery capacity, restricted flight time, the need to avoid obstacles, and ensuring communication security in unknown interference. Therefore, an intelligent and automatic UAV trajectory optimization technique is proposed. Through the DDQN algorithm, it improves training time and performance compared to traditional Q-learning algorithms.

The main contributions of the paper include:
1. Considering the influence of complex real environments, such as obstacles and interference, on 3D UAV trajectory planning.
2. Real-time perception and navigation of obstacles for UAVs by utilizing a memory buffer to accelerate decision-making.
3. Developing a DDQN-based UAV trajectory optimization algorithm, with rewards values based on scenario settings, which achieves faster convergence speed. The paper also provides comparative experiments of flight results and reinforcement learning algorithms under different scenario parameters, demonstrating the effectiveness and superiority of the method.

In conclusion, this paper aims to solve the efficient and secure data collection path planning problem for UAVs in a 3D environment using the DDQN algorithm, in order to address the challenges brought by the expansion of the Internet of Things network.