Abstract:As an essential resource management problem in network virtualization, virtual network embedding (VNE) aims to allocate the finite resources of physical network to sequentially arriving virtual network requests (VNRs) with different resource demands. Since this is an NP-hard combinatorial optimization problem, many efforts have been made to provide viable solutions. However, most existing approaches have either ignored the admission control of VNRs, which has a potential impact on long-term performances, or not fully exploited the temporal and topological features of the physical network and VNRs. In this paper, we propose a deep Hierarchical Reinforcement Learning approach to learn a joint Admission Control and Resource Allocation policy for VNE, named HRL-ACRA. Specifically, the whole VNE process is decomposed into an upper-level policy for deciding whether to admit the arriving VNR or not and a lower-level policy for allocating resources of the physical network to meet the requirement of VNR through the HRL approach. Considering the proximal policy optimization as the basic training algorithm, we also adopt the average reward method to address the infinite horizon problem of the upper-level agent and design a customized multi-objective intrinsic reward to alleviate the sparse reward issue of the lower-level agent. Moreover, we develop a deep feature-aware graph neural network to capture the features of VNR and physical network and exploit a sequence-to-sequence model to generate embedding actions iteratively. Finally, extensive experiments are conducted in various settings, and show that HRL-ACRA outperforms state-of-the-art baselines in terms of both the acceptance ratio and long-term average revenue. Our code is available at \url{<a class="link-external link-https" href="https://github.com/GeminiLight/hrl-acra" rel="external noopener nofollow">this https URL</a>}.

Intelligent Network Resource Management

RDAM: A Reinforcement Learning Based Dynamic Attribute Matrix Representation for Virtual Network Embedding

Network View and Cognitive Mechanism for Virtual Network Resource Management Based Intelligent

Network Resource Allocation Strategy Based on Deep Reinforcement Learning

Deep Reinforcement Learning (DRL)-based Resource Management in Software-Defined and Virtualized Vehicular Ad Hoc Networks

A novel reinforcement learning algorithm for virtual network embedding

Dynamic Reservation and Deep Reinforcement Learning based Autonomous Resource Management for wireless Virtual Networks

Reinforcement Learning Assisted Bandwidth Aware Virtual Network Resource Allocation

A Continuous-Decision Virtual Network Embedding Scheme Relying on Reinforcement Learning.

Meta-Hierarchical Reinforcement Learning (Mhrl)-Based Dynamic Resource Allocation for Dynamic Vehicular Networks

An RBF neural network-based dynamic virtual network embedding algorithm

Joint Admission Control and Resource Allocation of Virtual Network Embedding via Hierarchical Deep Reinforcement Learning

Space-Air-Ground Integrated Multi-Domain Network Resource Orchestration Based on Virtual Network Architecture: A DRL Method

Resource Governance in Networked Systems via Integrated Variational Autoencoders and Reinforcement Learning

Blocking Island Paradigm Enhanced Intelligent Coordinated Virtual Network Embedding Based on Deep Reinforcement Learning.

Intelligent Resource Allocation Management for Vehicles Network: an A3C Learning Approach.

A Node Probability-based Reinforcement Learning Framework for Virtual Network Embedding

Network Resource Allocation Algorithm Using Reinforcement Learning Policy-Based Network in a Smart Grid Scenario

Reinforcement-learning-based Wireless Resource Allocation

A neuro-fuzzy approach to self-management of virtual network resources

A Virtual Network Embedding Algorithm Based on RBF Neural Network