Abstract:As an essential resource management problem in network virtualization, virtual network embedding (VNE) aims to allocate the finite resources of physical network to sequentially arriving virtual network requests (VNRs) with different resource demands. Since this is an NP-hard combinatorial optimization problem, many efforts have been made to provide viable solutions. However, most existing approaches have either ignored the admission control of VNRs, which has a potential impact on long-term performances, or not fully exploited the temporal and topological features of the physical network and VNRs. In this paper, we propose a deep Hierarchical Reinforcement Learning approach to learn a joint Admission Control and Resource Allocation policy for VNE, named HRL-ACRA. Specifically, the whole VNE process is decomposed into an upper-level policy for deciding whether to admit the arriving VNR or not and a lower-level policy for allocating resources of the physical network to meet the requirement of VNR through the HRL approach. Considering the proximal policy optimization as the basic training algorithm, we also adopt the average reward method to address the infinite horizon problem of the upper-level agent and design a customized multi-objective intrinsic reward to alleviate the sparse reward issue of the lower-level agent. Moreover, we develop a deep feature-aware graph neural network to capture the features of VNR and physical network and exploit a sequence-to-sequence model to generate embedding actions iteratively. Finally, extensive experiments are conducted in various settings, and show that HRL-ACRA outperforms state-of-the-art baselines in terms of both the acceptance ratio and long-term average revenue. Our code is available at \url{<a class="link-external link-https" href="https://github.com/GeminiLight/hrl-acra" rel="external noopener nofollow">this https URL</a>}.

An Attention Based Deep Reinforcement Learning Method for Virtual Network Function Placement

Leveraging Deep Reinforcement Learning with Attention Mechanism for Virtual Network Function Placement and Routing

A Reinforcement Learning Approach for Placement of Stateful Virtualized Network Functions

Applications of Reinforcement Learning in Virtual Network Function Placement: A Survey.

Applicationsof ReinforcementLearningIn Virtual NetworkFunction Placement:A Survey

Network Resource Allocation Strategy Based on Deep Reinforcement Learning

Fair Virtual Network Function Scheduling with Deep Reinforcement Learning

Optimal VNF Placement Via Deep Reinforcement Learning in SDN/NFV-Enabled Networks.

Online Fault-tolerant VNF Chain Placement: A Deep Reinforcement Learning Approach

Combining Deep Reinforcement Learning with Graph Neural Networks for Optimal VNF Placement

Energy-Efficient and Interference-Aware VNF Placement with Deep Reinforcement Learning

DRL-SFCP: Adaptive Service Function Chains Placement with Deep Reinforcement Learning

Energy-delay-aware VNF scheduling: a reinforcement learning approach with hierarchical reward enhancement

DeepVNP:Virtual Network Placing with Deep Reinforcement Learning in Industrial IoT

Intelligent VNF Orchestration and Flow Scheduling Via Model-Assisted Deep Reinforcement Learning

A Reinforcement Learning Approach for Virtual Network Function Chaining and Sharing in Softwarized Networks

Parallel Placement of Virtualized Network Functions via Federated Deep Reinforcement Learning

Reliable Service Function Chain Deployment Method Based on Deep Reinforcement Learning

Joint Admission Control and Resource Allocation of Virtual Network Embedding via Hierarchical Deep Reinforcement Learning

GenAI Assistance for Deep Reinforcement Learning-based VNF Placement and SFC Provisioning in 5G Cores

A Continuous-Decision Virtual Network Embedding Scheme Relying on Reinforcement Learning.