Abstract:In recent years, the Internet of Vehicles (IoV) has been found to be of huge potential value in the promotion of the development of intelligent transportation systems (ITSs) and smart cities. However, the traditional scheme in IoV has difficulty in dealing with an uncertain environment, while reinforcement learning has the advantage of being able to deal with an uncertain environment. Spectrum resource allocation in IoV faces the uncertain environment in most cases. Therefore, this paper investigates the spectrum resource allocation problem by deep reinforcement learning after using spectrum sensing technology in the ITS, including the vehicle-to-infrastructure (V2I) link and the vehicle-to-vehicle (V2V) link. The spectrum resource allocation is modeled as a reinforcement learning-based multi-agent problem which is solved by using the soft actor critic (SAC) algorithm. Considered an agent, each V2V link interacts with the vehicle environment and makes a joint action. After that, each agent receives different observations as well as the same reward, and updates networks through the experiences from the memory. Therefore, during a certain time, each V2V link can optimize its spectrum allocation scheme to maximize the V2I capacity as well as increase the V2V payload delivery transmission rate. However, the number of SAC networks increases linearly as the number of V2V links increases, which means that the networks may have a problem in terms of convergence when there are an excessive number of V2V links. Consequently, a new algorithm, namely parameter sharing soft actor critic (PSSAC), is proposed to reduce the complexity for which the model is easier to converge. The simulation results show that both SAC and PSSAC can improve the V2I capacity and increase the V2V payload transmission success probability within a certain time. Specifically, these novel schemes have a 10 percent performance improvement compared with the existing scheme in the vehicular environment. Additionally, PSSAC has a lower complexity.

Deep-Reinforcement-Learning-Based AoI-Aware Resource Allocation for RIS-Aided IoV Networks

Air-to-Ground Integrated Internet of Vehicles Enhanced by LAPSs and RISs: Location, Power, and Phase Shift Optimization

Socially Aware V2X Networks with RIS: Joint Resource Optimization

Joint Phase Shift Design and Resource Management for a Non-Orthogonal Multiple Access-Enhanced Internet of Vehicle Assisted by an Intelligent Reflecting Surface-Equipped Unmanned Aerial Vehicle

Intelligent Reflective Surface Resource Allocation Algorithm Based on Deep Reinforcement Learning in Internet of Vehicles System

Optimal Resource Allocation for Integrated Sensing and Communications in Internet of Vehicles: A Deep Reinforcement Learning Approach

Parallel Surfaces: Service-Oriented V2X Communications for Autonomous Vehicles

A Deep Reinforcement Learning Scheme for Spectrum Sensing and Resource Allocation in ITS

AoI-Aware Joint Spectrum and Power Allocation for Internet of Vehicles: A Trust Region Policy Optimization-Based Approach.

Optimizing AoI in UAV-RIS Assisted IoT Networks: Off Policy vs. On Policy

Intelligent Surface Aided D2D-V2X System for Low-Latency and High-Reliability Communications.

QoS-Driven Spectrum Sharing for Reconfigurable Intelligent Surfaces (RISs) Aided Vehicular Networks

Intelligent Resource Management Based on Reinforcement Learning for Ultra-Reliable and Low-Latency IoV Communication Networks

Multi-Agent DRL-Based Task Offloading in Multiple RIS-Aided IoV Networks

UAV-RIS Assisted AoI Minimization in IoT Networks: A Reinforcement Learning Approach

Reconfigurable Intelligent Surface Aided Vehicular Edge Computing: Joint Phase-shift Optimization and Multi-User Power Allocation

Energy-Efficient Joint Resource Allocation With Reconfigurable Intelligent Surfaces in Symbiotic Radio Networks

DRL-Based Optimization for AoI and Energy Consumption in C-V2X Enabled IoV

Deep Reinforcement Learning Based Wireless Resource Allocation for V2X Communications

Tasks-oriented Joint Resource Allocation Scheme for the Internet of Vehicles with Sensing, Communication and Computing Integration

Adaptive Resource Allocation of Vehicles under Dynamic Environment