Abstract:This paper studies the multi-agent resource allocation problem in vehicular networks using non-orthogonal multiple access (NOMA) and network slicing. Vehicles want to broadcast multiple packets with heterogeneous quality-of-service (QoS) requirements, such as safety-related packets (e.g., accident reports) that require very low latency communication, while raw sensor data sharing (e.g., high-definition map sharing) requires high-speed communication. To ensure heterogeneous service requirements for different packets, we propose a network slicing architecture. We focus on a non-cellular network scenario where vehicles communicate by the broadcast approach via the direct device-to-device interface (i.e., sidelink communication). In such a vehicular network, resource allocation among vehicles is very difficult, mainly due to (i) the rapid variation of wireless channels among highly mobile vehicles and (ii) the lack of a central coordination point. Thus, the possibility of acquiring instantaneous channel state information to perform centralized resource allocation is precluded. The resource allocation problem considered is therefore very complex. It includes not only the usual spectrum and power allocation, but also coverage selection (which target vehicles to broadcast to) and packet selection (which network slice to use). This problem must be solved jointly since selected packets can be overlaid using NOMA and therefore spectrum and power must be carefully allocated for better vehicle coverage. To do so, we first provide a mathematical programming formulation and a thorough NP-hardness analysis of the problem. Then, we model it as a multi-agent Markov decision process. Finally, to solve it efficiently, we use a deep reinforcement learning (DRL) approach and specifically propose a deep Q learning (DQL) algorithm. The proposed DQL algorithm is practical because it can be implemented in an online and distributed manner. It is based on a cooperative learning strategy in which all agents perceive a common reward and thus learn cooperatively and distributively to improve the resource allocation solution through offline training. We show that our approach is robust and efficient when faced with different variations of the network parameters and compared to centralized benchmarks.

Spectrum-Energy-Efficient Mode Selection and Resource Allocation for Heterogeneous V2X Networks: A Federated Multi-Agent Deep Reinforcement Learning Approach

Deep Reinforcement Learning Based Mode Selection and Resource Allocation for Cellular V2X Communications

Multi-Agent Reinforcement Learning-Based Decentralized Spectrum Access in Vehicular Networks with Emergent Communication

Traffic Priority-Aware Multi-User Distributed Dynamic Spectrum Access: A Multi-Agent Deep RL Approach

Multi-Agent RL Enables Decentralized Spectrum Access in Vehicular Networks

Joint mode selection and resource allocation for cellular V2X communication using distributed deep reinforcement learning under 5G and beyond networks

Federated Multi-Agent Deep Reinforcement Learning for Resource Allocation of Vehicle-to-Vehicle Communications

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

Network slicing for vehicular communications: a multi-agent deep reinforcement learning approach

Decentralized Multi-Agent DQN-Based Resource Allocation for Heterogeneous Traffic in V2X Communications

Semantic-Aware Resource Allocation Based on Deep Reinforcement Learning for 5G-V2X HetNets

Multiagent Reinforcement Learning-Based Semi-Persistent Scheduling Scheme in C-V2X Mode 4

Meta-Reinforcement Learning Based Resource Allocation for Dynamic V2X Communications

Distributed Resource Allocation with Multi-Agent Deep Reinforcement Learning for 5G-V2V Communication

Deep Reinforcement Learning for Multi-Objective Resource Allocation in Multi-Platoon Cooperative Vehicular Networks

Load-Aware Distributed Resource Allocation for MF-TDMA Ad Hoc Networks: A Multi-Agent DRL Approach.

Multi-Agent Reinforcement Learning for Long-Term Network Resource Allocation through Auction: a V2X Application

An Intelligent Congestion Control Strategy in Heterogeneous V2X Based on Deep Reinforcement Learning

Deep Reinforcement Learning (DRL)-based Resource Management in Software-Defined and Virtualized Vehicular Ad Hoc Networks

Dynamic Spectrum Sharing Based on Deep Reinforcement Learning in Mobile Communication Systems