Kangwei Qi,Qiong Wu,Pingyi Fan,Nan Cheng,Wen Chen,Jiangzhou Wang,Khaled B. Letaief
Abstract:Reconfigurable Intelligent Surface (RIS) is a pivotal technology in communication, offering an alternative path that significantly enhances the link quality in wireless communication environments. In this paper, we propose a RIS-assisted internet of vehicles (IoV) network, considering the vehicle-to-everything (V2X) communication method. In addition, in order to improve the timeliness of vehicle-to-infrastructure (V2I) links and the stability of vehicle-to-vehicle (V2V) links, we introduce the age of information (AoI) model and the payload transmission probability model. Therefore, with the objective of minimizing the AoI of V2I links and prioritizing transmission of V2V links payload, we construct this optimization problem as an Markov decision process (MDP) problem in which the BS serves as an agent to allocate resources and control phase-shift for the vehicles using the soft actor-critic (SAC) algorithm, which gradually converges and maintains a high stability. A AoI-aware joint vehicular resource allocation and RIS phase-shift control scheme based on SAC algorithm is proposed and simulation results show that its convergence speed, cumulative reward, AoI performance, and payload transmission probability outperforms those of proximal policy optimization (PPO), deep deterministic policy gradient (DDPG), twin delayed deep deterministic policy gradient (TD3) and stochastic algorithms.
Machine Learning,Distributed, Parallel, and Cluster Computing,Networking and Internet Architecture,Signal Processing
What problem does this paper attempt to address?
The problem that this paper attempts to solve is in the Internet of Vehicles (IoV) network, how to minimize the Age of Information (AoI) of Vehicle - to - Infrastructure (V2I) links and improve the payload transmission success rate of Vehicle - to - Vehicle (V2V) links by optimizing resource allocation and the phase - shift matrix design of Reconfigurable Intelligent Surface (RIS). Specifically:
1. **Improve the information freshness of V2I links**: By introducing the AoI model, ensure that the communication between vehicles and infrastructure can update information in a timely manner and maintain the freshness of information.
2. **Ensure the reliability of V2V links**: By optimizing resource allocation, ensure the effective transmission of safety information between vehicles.
3. **Meet the challenges in high - mobility environments**: Due to the high - speed movement characteristics of vehicles, traditional optimization methods are difficult to achieve real - time and effective resource allocation. Therefore, this research proposes a method based on Deep Reinforcement Learning (DRL) to solve this problem.
4. **Use RIS technology to enhance communication performance**: By introducing RIS in the Internet of Vehicles, improve the signal propagation path, reduce information transmission delay, thereby further reducing AoI and increasing the transmission success rate.
### Optimization Objectives
The objective of this research can be formalized as an optimization problem, aiming to minimize the total AoI of all V2I links and maximize the successful transmission probability of V2V payloads. The specific mathematical expressions are as follows:
\[
\begin{aligned}
& \text{minimize} & & \frac{1}{N} \sum_{n = 1}^{N} A_m^n-\Pr\left\{\sum_{n = 1}^{N}\sum_{l = 1}^{L}x_{m,k}^n[l]BR_k^n[l]\Delta_n\geq D\right\}\\
& \text{subject to} & & R_m^n[l]\geq R_{th},\quad\forall m\in M,\forall k\in K,\\
& & & 0 < P_d^{k,n}< P_d^{\max},\quad\forall k\in K,\forall n\in N,\\
& & & x_{m,k}^n\in\{0, 1\},\quad\forall m\in M,\forall k\in K,\\
& & & \sum_mx_{m,k}^n\leq1,\quad\forall m\in M,\\
& & & 0\leq\theta_f^n\leq2\pi,\quad\forall f\in F,\forall n\in N.
\end{aligned}
\]
where:
- \(A_m^n\) represents the AoI of the \(m\)-th vehicle in the \(n\)-th time slot.
- \(R_k^n[l]\) represents the transmission rate of the \(k\)-th pair of V2V links on the \(l\)-th sub - channel in the \(n\)-th time slot.
- \(x_{m,k}^n[l]\) represents whether the CUE of the \(m\)-th vehicle allocates the \(l\)-th sub - channel to the \(k\)-th pair of DUEs in the \(n\)-th time slot.
- \(P_d^{k,n}\) represents the transmit power of the \(k\)-th pair of V2V links in the \(n\)-th time slot.
- \(\theta_f^n\) represents the phase shift of the \(f\)-th RIS element in the \(n\)-th time slot.
### Solutions
To achieve the above - mentioned optimization objectives, this research adopts a Deep Reinforcement Learning method based on the Soft Actor - Critic (SAC) algorithm. The SAC algorithm solves the problem through the following steps:
1. **State representation**: Define the system state \(s_n\), including the local channels of V2I and V2V links, the interference in the previous time slot, the remaining load of V2V links, and the information of V2I links