Abstract:In recent years, unmanned aerial vehicles (UAVs) are considered to be integrated into wireless communication systems because of their tremendous advantages in mobility, cost, maneuverability, etc. In some real UAV-assisted communication scenarios, the dynamics of the environment, such as the roaming of served users, make it hard to obtain an optimal trajectory before the UAV is dispatched. Implanting an intelligent control policy into UAVs for distributed task execution is necessary to complete the task. In this paper, a UAV trajectory design problem is investigated for an orthorgonal-frequency-division-multiplexing (OFDM) wireless sensor network, which is dynamic because mobile sensors may randomly roam within a certain range. The UAV is expected to balance task efficiency with the safety constraint with a pre-trained onboard control policy. Compared to prior works, this work requires the policy to adapt to randomly generated obstacle maps, and also assumes that the UAV has no prior knowledge of the obstacles before it is dispatched, which brings about challenges to the problem. The motivation comes from adversarial environments without the specific obstacle distribution beforehand, such as a disaster area. The problem is formulated as a constrained Markov decision process (CMDP) model, which incorporates the safety constraint compared to basic MDP. Due to the assumption of randomized obstacle distribution and lack of prior knowledge, existing algorithms for CMDP can not be applied directly. To tackle this issue, we enhance reinforcement learning (RL) algorithm with a safety control mechanism to derive our novel safe reinforcement learning (Safe RL) algorithm, which is based on the framework of Lagrangian method. Compared to former algorithms about CMDP, our algorithm eliminates the premise that the safety model is known, the agent is able to learn safety judgement from scratch through its interactions with the environment. Simulation results demonstrate that our proposed algorithm outperforms the benchmark algorithm under the problem’s setup.

Online Trajectory Optimization for the UAV-Enabled Base Station Multicasting System Based on Reinforcement Learning

Online Trajectory Optimization for the UAV-Mounted Base Stations

Trajectory Optimization for Completion Time Minimization in UAV-Enabled Multicasting

Joint Trajectory and Power Optimization in Multi-Type UAVs Network with Mean Field Q-Learning.

Trajectory Design for Completion Time Minimization in UAV-Enabled Multicasting

Non-position-based UAV Trajectory Optimization for Coverage Maximization

Joint Resource Allocation and Trajectory Optimization for UAV-Aided Relay Networks

UAV-Enabled Aerial Base Station (BS) I/III: Joint Trajectory and Communication Design for Multi-UAV Enabled Wireless Networks

Three-Dimensional Trajectory and Resource Allocation Optimization in Multi-Unmanned Aerial Vehicle Multicast System: A Multi-Agent Reinforcement Learning Method

Distributed UAV-BSs Trajectory Optimization for User-Level Fair Communication Service with Multi-Agent Deep Reinforcement Learning

Trajectory and Beamforming Vector Optimization for Multi-UAV Multicast Network.

Safety Constrained Trajectory Optimization for Completion Time Minimization for UAV Communications

Delay-Tolerant UAV-Assisted Communication: Online Trajectory Design and User Association

Joint Trajectory and Communication Design for UAV-Enabled Multiple Access

Reinforcement Learning Based Joint Trajectory Design and Resource Allocation for RIS-aided UAV Multicast Networks

The Transmission Scheme for UAV MIMO System Based on Reinforcement Learning

Trajectory Design and Bandwidth Assignment for UAVs-enabled Communication Network with Multi - Agent Deep Reinforcement Learning.

Trajectory Optimization and Resource Allocation for UAV Base Stations under In-Band Backhaul Constraint

Joint Optimization of Access and Backhaul Links for UAVs Based on Reinforcement Learning

Bayesian Optimization Enhanced Deep Reinforcement Learning for Trajectory Planning and Network Formation in Multi-UAV Networks

Joint UAV trajectory and communication design with heterogeneous multi-agent reinforcement learning