Abstract:Network function virtualization (NFV) in elastic optical datacenter interconnections (EO-DCIs) enables flexible and timely deployment of network services. However, as the service provisioning of virtual network function service chains (vNF-SCs) in an EO-DCI needs to orchestrate the allocations of IT resources in datacenters (DCs) and spectrum resources on fiber links dynamically, it is a complex and challenging problem. In this work, we model the problem as a Markov decision process (MDP), and propose a hierarchical deep reinforcement learning (DRL) model based on graph neural network (GNN), namely, HRLOrch, to tackle it. To ensure its universality and scalability, we design the policy neural network (NN) in HRLOrch based on a GNN. As the GNN-based policy NN can operate on the graph-structured network state of an EO-DCI directly, it can adapt to an arbitrary EO-DCI topology without any structural changes. Then, through analysis, we find that the EO-DCI is a sparse reward environment if we want to train a DRL model to minimize the blocking probability of vNF-SCs in it directly. To address this issue, we design a hierarchical DRL with lower-level and upper-level models to improve the convergence performance of training. Specifically, we make the lower-level DRL optimize the provisioning scheme of each vNF-SC to minimize its resource usage, while the upper-level one coordinates the provisioning of all the active vNF-SCs to minimize the overall blocking probability. Hence, the lower-level and upper-level DRL models operate cooperatively in the training to optimize the dynamic provisioning of vNF-SCs. Our simulations demonstrate the universality and scalability of HRLOrch, and confirm that it can outperform the existing algorithms for vNF-SC provisioning in an EO-DCI.

Energy-Efficient Graph Reinforced vNFC Deployment in Elastic Optical Inter-DC Networks

Energy-Efficient Virtual Network Embedding: A Deep Reinforcement Learning Approach Based on Graph Convolutional Networks

Cost-Efficient Virtual Network Function Graph (vnfg) Provisioning in Multidomain Elastic Optical Networks

5G Ultradense Cellular-Network-Based Edge Demand Response: Energy Consumption Reduction

ENCG-DRL for Multicast Service Oriented Energy-Efficient DU-CU Deployment and RMSA in EON-enabled RAN

GNN-Based Hierarchical Deep Reinforcement Learning for NFV-Oriented Online Resource Orchestration in Elastic Optical DCIs

Orchestrating Tree-Type VNF Forwarding Graphs in Inter-DC Elastic Optical Networks

Virtual-Pod-Assisted Routing and Resource Assignment in Elastic All-Optical Intra-Datacenter Networks

Energy-delay-aware VNF scheduling: a reinforcement learning approach with hierarchical reward enhancement

Energy-aware Service Function Chaining Embedding in NFV Networks

Deep Reinforcement Learning for Resource Demand Prediction and Virtual Function Network Migration in Digital Twin Network

Energy-efficient URLLC Service Provisioning in Softwarization-Based Networks

Service Provisioning with Energy-Aware Regenerator Allocation in Multi-Domain EONs

Deploying Network Functions for Multiaccess Edge-IoT with Deep Reinforcement Learning

Deep-learning-assisted Network Orchestration for On-Demand and Cost-Effective VNF Service Chaining in Inter-Dc Elastic Optical Networks

Energy-Efficient Provisioning for Service Function Chains to Support Delay-Sensitive Applications in Network Function Virtualization

Energy-Aware Virtual Machine Management in Inter-datacenter Networks over Elastic Optical Infrastructure

Network Service Reconfiguration in Hybrid Optical/Electrical Datacenter Networks

Network Function Virtualization in Elastic Optical Networks

GreenNFV: Energy-Efficient Network Function Virtualization with Service Level Agreement Constraints

Deep-NFVOrch: Leveraging Deep Reinforcement Learning to Achieve Adaptive Vnf Service Chaining in DCI-EONs