Abstract:In recent years, unmanned aerial vehicles (UAVs) are considered to be integrated into wireless communication systems because of their tremendous advantages in mobility, cost, maneuverability, etc. In some real UAV-assisted communication scenarios, the dynamics of the environment, such as the roaming of served users, make it hard to obtain an optimal trajectory before the UAV is dispatched. Implanting an intelligent control policy into UAVs for distributed task execution is necessary to complete the task. In this paper, a UAV trajectory design problem is investigated for an orthorgonal-frequency-division-multiplexing (OFDM) wireless sensor network, which is dynamic because mobile sensors may randomly roam within a certain range. The UAV is expected to balance task efficiency with the safety constraint with a pre-trained onboard control policy. Compared to prior works, this work requires the policy to adapt to randomly generated obstacle maps, and also assumes that the UAV has no prior knowledge of the obstacles before it is dispatched, which brings about challenges to the problem. The motivation comes from adversarial environments without the specific obstacle distribution beforehand, such as a disaster area. The problem is formulated as a constrained Markov decision process (CMDP) model, which incorporates the safety constraint compared to basic MDP. Due to the assumption of randomized obstacle distribution and lack of prior knowledge, existing algorithms for CMDP can not be applied directly. To tackle this issue, we enhance reinforcement learning (RL) algorithm with a safety control mechanism to derive our novel safe reinforcement learning (Safe RL) algorithm, which is based on the framework of Lagrangian method. Compared to former algorithms about CMDP, our algorithm eliminates the premise that the safety model is known, the agent is able to learn safety judgement from scratch through its interactions with the environment. Simulation results demonstrate that our proposed algorithm outperforms the benchmark algorithm under the problem’s setup.

Safety Constrained Trajectory Optimization for Completion Time Minimization for UAV Communications

Trajectory Optimization for UAV Emergency Communication with Limited User Equipment Energy: A Safe-DQN Approach.

Interpretable and Secure Trajectory Optimization for UAV-Assisted Communication

Three-Dimensional Trajectory Design for Multi-User MISO UAV Communications: A Deep Reinforcement Learning Approach

Energy-Efficient Multi-UAVs Cooperative Trajectory Optimization for Communication Coverage: An MADRL Approach

Online Trajectory Optimization for the UAV-Enabled Base Station Multicasting System Based on Reinforcement Learning

Model Predictive Control Enabled UAV Trajectory Optimization and Secure Resource Allocation

Online Trajectory Optimization for the UAV-Mounted Base Stations

Optimal Transmission Control and Learning-Based Trajectory Design for UAV-Assisted Detection and Communication

Learning to Navigate for Secure UAV Communication

Learning-Based UAV Trajectory Optimization with Collision Avoidance and Connectivity Constraints

Joint Resource Allocation and Trajectory Optimization for Completion Time Minimization for Energy-Constrained UAV Communications

Three-dimensional deep reinforcement learning for trajectory and resource optimization in UAV communication systems

Trajectory Design for Multi-UAV-Enabled Wireless Powered Communication Networks: A Multi-Agent DRL Approach

Intelligent Trajectory Design in UAV-Aided Communications with Reinforcement Learning

Robust 3D-Trajectory and Time Switching Optimization for Dual-UAV-Enabled Secure Communications

Three-Dimension Trajectory Design for Multi-UAV Wireless Network With Deep Reinforcement Learning

Deep Reinforcement Learning Based Trajectory Design and Resource Allocation for UAV-Assisted Communications

Joint Trajectory and Scheduling Optimization for Age of Synchronization Minimization in UAV-Assisted Networks with Random Updates

Research on Joint Optimization of Task Offloading and UAV Trajectory in Mobile Edge Computing Considering Communication Cost Based on Safe Reinforcement Learning

Cellular UAV-to-Device Communications: Trajectory Design and Mode Selection by Multi-agent Deep Reinforcement Learning