Abstract:In recent years, unmanned aerial vehicles (UAVs) are considered to be integrated into wireless communication systems because of their tremendous advantages in mobility, cost, maneuverability, etc. In some real UAV-assisted communication scenarios, the dynamics of the environment, such as the roaming of served users, make it hard to obtain an optimal trajectory before the UAV is dispatched. Implanting an intelligent control policy into UAVs for distributed task execution is necessary to complete the task. In this paper, a UAV trajectory design problem is investigated for an orthorgonal-frequency-division-multiplexing (OFDM) wireless sensor network, which is dynamic because mobile sensors may randomly roam within a certain range. The UAV is expected to balance task efficiency with the safety constraint with a pre-trained onboard control policy. Compared to prior works, this work requires the policy to adapt to randomly generated obstacle maps, and also assumes that the UAV has no prior knowledge of the obstacles before it is dispatched, which brings about challenges to the problem. The motivation comes from adversarial environments without the specific obstacle distribution beforehand, such as a disaster area. The problem is formulated as a constrained Markov decision process (CMDP) model, which incorporates the safety constraint compared to basic MDP. Due to the assumption of randomized obstacle distribution and lack of prior knowledge, existing algorithms for CMDP can not be applied directly. To tackle this issue, we enhance reinforcement learning (RL) algorithm with a safety control mechanism to derive our novel safe reinforcement learning (Safe RL) algorithm, which is based on the framework of Lagrangian method. Compared to former algorithms about CMDP, our algorithm eliminates the premise that the safety model is known, the agent is able to learn safety judgement from scratch through its interactions with the environment. Simulation results demonstrate that our proposed algorithm outperforms the benchmark algorithm under the problem’s setup.

Optimal control-based UAV path planning with dynamically-constrained TSP with neighborhoods

3D Path Planning of the Solar Powered UAV in the Urban-mountainous Environment with Multi-objective and Multi-constraint Based on the Enhanced Sparrow Search Algorithm Incorporating the Levy Flight Strategy

A Multi-Objective Coverage Path Planning Algorithm for UAVs to Cover Spatially Distributed Regions in Urban Environments

UAV Trajectory Planning for Data Collection from Time-Constrained IoT Devices

Trajectory Optimization for Cellular-Enabled UAV with Connectivity and Battery Constraints

Constrained Heterogeneous Vehicle Path Planning for Large-area Coverage

An Integrated Geometric Obstacle Avoidance and Genetic Algorithm TSP Model for UAV Path Planning

UAV Network Path Planning and Optimization Using a Vehicle Routing Model

UAV Trajectory Planning with Interference Awareness in UAV-Enabled Time-Constrained Data Collection Systems

3-D real-time smooth path planning for UAV based on TLD model

A Path Planning Method for Collaborative Coverage Monitoring in Urban Scenarios

A new method for unmanned aerial vehicle path planning in complex environments

On Optimal Path Planning for UAV Based Patrolling in Complex 3D Topographies

Optimal Trajectory Planning for UAV-relayed Dynamic Spectrum Access

Towards Autonomous Operation of UAVs Using Data-Driven Target Tracking and Dynamic, Distributed Path Planning Methods

Reinforcement Learning-Based Collision Avoidance and Optimal Trajectory Planning in UAV Communication Networks

Trajectory Planning for Unmanned Aerial Vehicles in Complicated Urban Environments: A Control Network Approach

Time-optimal UAV Trajectory Planning for 3D Urban Structure Coverage

Enhanced Teaching-Learning-based Optimization for 3D Path Planning of Multicopter UAVs

Safety Constrained Trajectory Optimization for Completion Time Minimization for UAV Communications

Real-time and Coordinated UAV Path Planning for Road Traffic Surveillance: A Penalty-based Boundary Intersection Approach