Abstract:Intelligent Transportation Systems (ITS) are gaining substantial attention owing to the great benefits offered to the vehicle users. In ITS paradigm, content data is normally obtained from road side units (RSUs). However, in some scenarios, terrestrial networks are partially/temporarily out-of-service. Unmanned Aerial Vehicle (UAV) or drone cells are expected to be one of the pillars of future networks to assist the vehicular networks in such scenarios. To this end, we propose a collaborative framework between UAVs and in-service RSUs to partial service vehicles. Our objective is to maximize the amount of downloaded contents to vehicles while considering the dynamic nature of the network. Motivated by the success of machine learning (ML) techniques particularly deep Reinforcement learning in solving complex problems, we formulate the scheduling and content management policy problem as a Markov Decision Process (MDP) where the system state space considers the vehicular network dynamics. Proximal Policy Optimization (PPO) is utilized to govern the content decisions in the vehicular network. The simulation-based results show that during the mission time, the proposed algorithm learns the vehicular environment and its dynamics to handle the complex action space.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: in the Intelligent Transportation System (ITS), how to maximize the amount of content transmitted to vehicles and ensure the Quality of Service (QoS) through the cooperation of Unmanned Aerial Vehicles (UAVs) and Road - side Units (RSUs) in the case of partial or temporary loss of ground network services. Specifically, the paper proposes a collaborative framework that utilizes the cooperation between UAVs and RSUs to provide content services for vehicles. This framework aims to deal with the dynamic characteristics of vehicle networks and optimize the scheduling and content management strategies through the Proximal Policy Optimization (PPO) algorithm in deep reinforcement learning. ### Problem Background 1. **Requirements of Intelligent Transportation System (ITS)** - ITS provides a great deal of convenience for vehicle users, but it depends on the content data provided by RSUs. - In some cases, the ground network may be partially or temporarily unavailable, resulting in the inability to guarantee the quality of service. 2. **Role of Unmanned Aerial Vehicles (UAVs)** - UAVs are regarded as an important part of future networks because of their flexibility, deployability and agility, especially when the ground network is unavailable and can assist vehicle networks. 3. **Existing Challenges** - The vehicle network environment has intermittent connections and constantly changing topologies, which are prone to cause delays and incomplete processes. - RSUs may not be able to fully meet the needs of all vehicles or may not work properly due to hardware failures or damages. ### Proposed Solution The paper proposes a collaborative framework in which UAVs cooperate with RSUs. Based on caching, UAVs can obtain, carry and forward content without accessing the backhaul link. Specific measures include: - **Content Caching Mechanism**: RSUs are responsible for storing part of the content on passing vehicles, and these vehicles then transmit the content to UAVs. - **Intelligent Decision - making**: In order to operate this framework effectively, the author proposes using deep reinforcement learning (especially the PPO algorithm) for intelligent decision - making to maximize the goals of RSU operators. ### Mathematical Modeling In order to achieve the above - mentioned goals, the paper formalizes the problem as a Markov Decision Process (MDP) and solves it through the PPO algorithm. Key elements such as the state space, action space, state transition probability and reward function involved in the model are defined in detail. ### Experimental Results Through simulation verification, the experimental results show that the proposed PPO algorithm performs excellently in dealing with complex action spaces, can effectively learn the vehicle environment and its dynamic changes, and is thus significantly superior to the random method. ### Summary This paper proposes a content distribution framework based on the cooperation of UAVs and RSUs, aiming to improve the content transmission efficiency and service quality in the Intelligent Transportation System. By introducing deep reinforcement learning techniques, especially the PPO algorithm, this framework can make optimal decisions in a dynamic environment, thereby improving the overall performance. ### Key Formulas 1. **Channel State Probability** \[ p(S^n_{R \to i}=\text{LoS})=\frac{1}{1 + \eta_1 e^{(-\eta_2(\theta^n_{R \to i}-\eta_1))}}, \quad \forall i\in I_n, n \] where \[ \theta^n_{R \to i}=\frac{180}{\pi}\arctan\left(\frac{z_R}{d^n_{R \to i}}\right) \] 2. **Channel Gain** \[ h^n_{R \to i}= \begin{cases} (D^n_{R \to i})^{-\beta_1}, & S^n_{R \to i}=\text{LoS}\\ \beta_2(D^n_{R \to i})^{-\beta_1}, & \text{otherwise} \end{cases} \] 3. **Instantaneous Throughput** \[ T^n_{R \to i}=W\log\left(1 + \fra

Cooperative content delivery in UAV-RSU assisted vehicular networks

UAV-Assisted Content Delivery in Intelligent Transportation Systems-Joint Trajectory Planning and Cache Management

Multi-UAV Multi-RIS QoS-Aware Aerial Communication Systems using DRL and PSO

Relay in the Sky: A UAV-Aided Cooperative Data Dissemination Scheduling Strategy in VANETs

UAV-Assisted Data Dissemination Scheduling in VANETs

Deep Reinforcement Learning for Unmanned Aerial Vehicle-Assisted Vehicular Networks

Dense Multi-Agent Reinforcement Learning Aided Multi-UAV Information Coverage for Vehicular Networks

Traffic Learning and Proactive UAV Trajectory Planning for Data Uplink in Markovian IoT Models

Deep Reinforcement Learning for Joint Trajectory Planning, Transmission Scheduling, and Access Control in UAV-Assisted Wireless Sensor Networks

Muti-Agent Proximal Policy Optimization For Data Freshness in UAV-assisted Networks

Multi-Agent Reinforcement Learning for Offloading Cellular Communications with Cooperating UAVs

Line-of-Sight-Based Coordinated Channel Resource Allocation Management in UAV-Assisted Vehicular Ad Hoc Networks

Neural Combinatorial Deep Reinforcement Learning for Age-optimal Joint Trajectory and Scheduling Design in UAV-assisted Networks

Learning-Based Cooperative Aerial and Ground Vehicle Routing for Emergency Communications

Multi-UAV-assisted covert communications for secure content delivery in Internet of Things

Multi-UAV Hierarchical Intelligent Traffic Offloading Network Optimization Based on Deep Federated Learning

On-Demand Routing for Urban VANETs using Cooperating UAVs

Deep Reinforcement Learning-Based Multirestricted Dynamic-Request Transportation Framework

Multiple UAVs collaborative traffic monitoring with intention-based communication

UAV-Aided Data Dissemination Protocol with Dynamic Trajectory Scheduling in VANETs.

UAV-Assisted Enhanced Coverage and Capacity in Dynamic MU-mMIMO IoT Systems: A Deep Reinforcement Learning Approach