Abstract:Service function chaining (SFC) focuses mainly on deploying various network functions in geographically distributed data centers and providing interconnect routing among them. Traditional (convex optimization-based) SFC algorithms exhibit some disadvantages on the scalability and accuracy. Recently, researches have shown the effectiveness of deep reinforcement learning (DRL) in the field of SFC. However, current DRL-based algorithms possess an extremely large action space, which leads to slow convergence and poor scalability. Some researchers relieve this issue by reformulating the SFC problem, which usually results in low utilization and high cost. To address this issue, we develop a hybrid DRL-based framework which decouples the VNF deployment and flow routing into different modules. In the proposed framework, a DRL agent is only responsible for learning the policy of VNF deployment. We customize the structure of the agent base on deep deterministic policy gradient (DDPG) and adopt several techniques to improve the learning efficiency, such as adaptive parameter noise, wolpertinger policy, and prioritized experience replay. The flow routing is conducted in a game-based module (GBM). We design a decentralized routing algorithm for the GBM to address the scalability. The end-to-end latency of flows is minimized while the resource capacity and location constraints are satisfied. During the learning process of the proposed framework, the DRL agent improves its deployment policy with the reward from the GBM (the value of reward depends on flow routing). Thus, the VNF deployment and flow routing are still jointly optimized. Compared to existing DRL-learning algorithms, the proposed hybrid DRL framework can achieve a lower cost since 1) the action space is significantly reduced due to flow routing decoupling; 2) the flow routing procedure is more efficient (the GBM adopts model-based information, e.g., the gradient). Through trace-driven simulations, we show the efficiency of our algorithm compared to existing DRL-based algorithms.

Deep-Nfvorch: Deep Reinforcement Learning Based Service Framework For Adaptive Vnf Service Chaining In Idc-Eons

Deep-NFVOrch: Leveraging Deep Reinforcement Learning to Achieve Adaptive Vnf Service Chaining in DCI-EONs

Deep-learning-assisted Network Orchestration for On-Demand and Cost-Effective VNF Service Chaining in Inter-Dc Elastic Optical Networks

DRL-Deploy:Adaptive Service Function Chains Deployment with Deep Reinforcement Learning

Designing Deep Learning Model for Accurate Vnf Service Chain Pre-deployment in Inter-DC EONs

GNN-Based Hierarchical Deep Reinforcement Learning for NFV-Oriented Online Resource Orchestration in Elastic Optical DCIs

Dynamic Service Function Chain Orchestration for NFV/MEC-Enabled IoT Networks: A Deep Reinforcement Learning Approach

DRL-Based Network Orchestration to Realize Cooperative, Distributed and Tenant-Driven Virtual Network Slicing

Scalable Orchestration of Service Function Chains in NFV-Enabled Networks: A Federated Reinforcement Learning Approach

Orchestrating data-intensive vNF service chains in inter-DC elastic optical networks

Service Function Chain Embedding for NFV-Enabled IoT Based on Deep Reinforcement Learning.

On Incentive-Driven VNF Service Chaining in Inter-Datacenter Elastic Optical Networks: A Hierarchical Game-Theoretic Mechanism.

Opti-DeepRoute: A Topology-Adaptive Deep Reinforcement Learning Based Service Provisioning Framework for Elastic Optical Network

DRL-QOR: Deep Reinforcement Learning-Based QoS/QoE-Aware Adaptive Online Orchestration in NFV-Enabled Networks

Double-Agent Reinforced Vnfc Deployment in EONs for Cloud-Edge Computing

A Hybrid Learning Framework for Service Function Chaining Across Geo-Distributed Data Centers.

Dynamic Service Function Chain Embedding for NFV-Enabled IoT: A Deep Reinforcement Learning Approach

Service Function Chain Embedding Meets Machine Learning: Deep Reinforcement Learning Approach

Service Function Chain Deployment Using Deep Q Learning and Tidal Mechanism

Intelligent VNF Orchestration and Flow Scheduling Via Model-Assisted Deep Reinforcement Learning

DeepCoop: Leveraging Cooperative DRL Agents to Achieve Scalable Network Automation for Multi-Domain SD-EONs