Abstract:Unmanned aerial vehicles (UAVs) are envisioned to complement the 5G communication infrastructure in future smart cities. Hot spots easily appear in road intersections, where effective communication among vehicles is challenging. UAVs may serve as relays with the advantages of low price, easy deployment, line-of-sight links, and flexible mobility. In this paper, we study a UAV-assisted vehicular network where the UAV jointly adjusts its transmission control (power and channel) and 3D flight to maximize the total throughput. First, we formulate a Markov decision process (MDP) problem by modeling the mobility of the UAV/vehicles and the state transitions. Secondly, we solve the target problem using a deep reinforcement learning method, namely, the deep deterministic policy gradient (DDPG), and propose three solutions with different control objectives. Deep reinforcement learning methods obtain the optimal policy through the interactions with the environment without knowing the environment variables. Considering that environment variables in our problem are unknown and unmeasurable, we choose a deep reinforcement learning method to solve it. Moreover, considering the energy consumption of 3D flight, we extend the proposed solutions to maximize the total throughput per unit energy. To encourage or discourage the UAV's mobility according to its prediction, the DDPG framework is modified, where the UAV adjusts its learning rate automatically. Thirdly, in a simplified model with small state space and action space, we verify the optimality of proposed algorithms. Comparing with two baseline schemes, we demonstrate the effectiveness of proposed algorithms in a realistic model.

UAV Trajectory Planning for Data Collection in Internet of Vehicles: A Reinforcement Learning Approach

Research on the UAV-aided Data Collection and Trajectory Design Based on the Deep Reinforcement Learning

3D UAV Trajectory and Data Collection Optimisation via Deep Reinforcement Learning

DRL-UTPS: DRL-Based Trajectory Planning for Unmanned Aerial Vehicles for Data Collection in Dynamic IoT Network

Trajectory Design for UAV-Based Internet of Things Data Collection: A Deep Reinforcement Learning Approach

Trajectory Planning for UAV-Assisted Data Collection in IoT Network: A Double Deep Q Network Approach

UAV Path Planning for Wireless Data Harvesting: A Deep Reinforcement Learning Approach

UAV Trajectory Optimization for Large-Scale and Low-Power Data Collection: An Attention-Reinforced Learning Scheme

Deep Reinforcement Learning for Fresh Data Collection in UAV-assisted IoT Networks

Deep Reinforcement Learning for Joint Trajectory Planning, Transmission Scheduling, and Access Control in UAV-Assisted Wireless Sensor Networks

Traffic Learning and Proactive UAV Trajectory Planning for Data Uplink in Markovian IoT Models

Deep Reinforcement Learning-Driven UAV Data Collection Path Planning: A Study on Minimizing AoI

Joint Cluster Head Selection and Trajectory Planning in UAV-Aided IoT Networks by Reinforcement Learning with Sequential Model

Deep Reinforcement Learning for Unmanned Aerial Vehicle-Assisted Vehicular Networks

Deep Reinforcement Learning for Efficient Data Collection in UAV-Aided Internet of Things

Joint Speed Control and Energy Replenishment Optimization for UAV-assisted IoT Data Collection with Deep Reinforcement Transfer Learning

UAV-Enabled Data Collection for IoT Networks via Rainbow Learning

Learning-Based UAV Path Planning for Data Collection with Integrated Collision Avoidance

3D-Trajectory and Phase-Shift Design for RIS-Assisted UAV Systems Using Deep Reinforcement Learning

Model-aided Deep Reinforcement Learning for Sample-efficient UAV Trajectory Design in IoT Networks

The UAV Trajectory Optimization for Data Collection from Time-Constrained IoT Devices: A Hierarchical Deep Q-Network Approach