Abstract:Service function chaining (SFC) focuses mainly on deploying various network functions in geographically distributed data centers and providing interconnect routing among them. Traditional (convex optimization-based) SFC algorithms exhibit some disadvantages on the scalability and accuracy. Recently, researches have shown the effectiveness of deep reinforcement learning (DRL) in the field of SFC. However, current DRL-based algorithms possess an extremely large action space, which leads to slow convergence and poor scalability. Some researchers relieve this issue by reformulating the SFC problem, which usually results in low utilization and high cost. To address this issue, we develop a hybrid DRL-based framework which decouples the VNF deployment and flow routing into different modules. In the proposed framework, a DRL agent is only responsible for learning the policy of VNF deployment. We customize the structure of the agent base on deep deterministic policy gradient (DDPG) and adopt several techniques to improve the learning efficiency, such as adaptive parameter noise, wolpertinger policy, and prioritized experience replay. The flow routing is conducted in a game-based module (GBM). We design a decentralized routing algorithm for the GBM to address the scalability. The end-to-end latency of flows is minimized while the resource capacity and location constraints are satisfied. During the learning process of the proposed framework, the DRL agent improves its deployment policy with the reward from the GBM (the value of reward depends on flow routing). Thus, the VNF deployment and flow routing are still jointly optimized. Compared to existing DRL-learning algorithms, the proposed hybrid DRL framework can achieve a lower cost since 1) the action space is significantly reduced due to flow routing decoupling; 2) the flow routing procedure is more efficient (the GBM adopts model-based information, e.g., the gradient). Through trace-driven simulations, we show the efficiency of our algorithm compared to existing DRL-based algorithms.

OptRec: an Efficient DRL-Based SFC Reconfiguration Optimization Framework in Programmable Networks

Scheduling Reconfigurable Service Function Chain

DRL-D: Revenue-Aware Online Service Function Chain Deployment Via Deep Reinforcement Learning

Multi-objective Optimization Service Function Chain Placement Algorithm Based on Reinforcement Learning

A Multi-Agent Reinforcement Learning Scheme for SFC Placement in Edge Computing Networks

A Hybrid Learning Framework for Service Function Chaining Across Geo-Distributed Data Centers.

A New Realistic Platform for Benchmarking and Performance Evaluation of DRL-Driven and Reconfigurable SFC Provisioning Solutions

Online VNF Chaining and Predictive Scheduling: Optimality and Trade-offs

Reliable Service Function Chain Deployment Method Based on Deep Reinforcement Learning

SFC active reconfiguration based on user mobility and resource demand prediction in dynamic IoT-MEC networks

Reliability-assured Service Function Chain Migration Strategy in Edge Networks Using Deep Reinforcement Learning

Scalable Orchestration of Service Function Chains in NFV-Enabled Networks: A Federated Reinforcement Learning Approach

Automatically Reconfigurable Optical Network for HPC System Based on Deep Reinforcement Learning

Service Function Chain Embedding Meets Machine Learning: Deep Reinforcement Learning Approach

A Deep Reinforcement Learning-Based Algorithm for Reliability-Aware Multi-Domain Service Deployment in Smart Ecosystems

Scalable Hybrid Switching-Driven Software Defined Networking Issue: From the Perspective of Reinforcement Learning Standpoint

External Memories of PDP Switches for In-Network Implementable Functions Placement: Deep Learning Based Reconfiguration of SFCs

NFVdeep

Fault Tolerance Oriented SFC Optimization in SDN/NFV-Enabled Cloud Environment Based on Deep Reinforcement Learning

A Q-Learning-Based Approach For Deploying Dynamic Service Function Chains

RuleDRL: Reliability-Aware SFC Provisioning with Bounded Approximations in Dynamic Environments