Abstract:As an essential resource management problem in network virtualization, virtual network embedding (VNE) aims to allocate the finite resources of physical network to sequentially arriving virtual network requests (VNRs) with different resource demands. Since this is an NP-hard combinatorial optimization problem, many efforts have been made to provide viable solutions. However, most existing approaches have either ignored the admission control of VNRs, which has a potential impact on long-term performances, or not fully exploited the temporal and topological features of the physical network and VNRs. In this paper, we propose a deep Hierarchical Reinforcement Learning approach to learn a joint Admission Control and Resource Allocation policy for VNE, named HRL-ACRA. Specifically, the whole VNE process is decomposed into an upper-level policy for deciding whether to admit the arriving VNR or not and a lower-level policy for allocating resources of the physical network to meet the requirement of VNR through the HRL approach. Considering the proximal policy optimization as the basic training algorithm, we also adopt the average reward method to address the infinite horizon problem of the upper-level agent and design a customized multi-objective intrinsic reward to alleviate the sparse reward issue of the lower-level agent. Moreover, we develop a deep feature-aware graph neural network to capture the features of VNR and physical network and exploit a sequence-to-sequence model to generate embedding actions iteratively. Finally, extensive experiments are conducted in various settings, and show that HRL-ACRA outperforms state-of-the-art baselines in terms of both the acceptance ratio and long-term average revenue. Our code is available at \url{<a class="link-external link-https" href="https://github.com/GeminiLight/hrl-acra" rel="external noopener nofollow">this https URL</a>}.

Energy-delay-aware VNF scheduling: a reinforcement learning approach with hierarchical reward enhancement

Fair Virtual Network Function Scheduling with Deep Reinforcement Learning

A reinforcement learning method for scheduling service function chains with multi-resource constraints

Age of Information Aware VNF Scheduling in Industrial IoT Using Deep Reinforcement Learning

Learning to schedule multi-NUMA virtual machines via reinforcement learning

Energy-Efficient and Interference-Aware VNF Placement with Deep Reinforcement Learning

Joint Admission Control and Resource Allocation of Virtual Network Embedding via Hierarchical Deep Reinforcement Learning

Network Resource Allocation Strategy Based on Deep Reinforcement Learning

Optimal Energy System Scheduling Using A Constraint-Aware Reinforcement Learning Algorithm

Parallel Placement of Virtualized Network Functions via Federated Deep Reinforcement Learning

A Reliable Resource Scheduling for Network Function Virtualization.

The Optimized Reinforcement Learning Approach to Run-Time Scheduling in Data Center.

Reliable Service Function Chain Deployment Method Based on Deep Reinforcement Learning

GreenNFV: Energy-Efficient Network Function Virtualization with Service Level Agreement Constraints

GNN-Based Hierarchical Deep Reinforcement Learning for NFV-Oriented Online Resource Orchestration in Elastic Optical DCIs

Real-Time Virtual Machine Scheduling in Industry IoT Network: A Reinforcement Learning Method

DRL-D: Revenue-Aware Online Service Function Chain Deployment Via Deep Reinforcement Learning

Enhancing network function parallelism in mobile edge computing using Deep Reinforcement Learning

FlagVNE: A Flexible and Generalizable Reinforcement Learning Framework for Network Resource Allocation

Energy-Efficient Graph Reinforced vNFC Deployment in Elastic Optical Inter-DC Networks

Flexible job shop scheduling via deep reinforcement learning with meta-path-based heterogeneous graph neural network