Abstract:Service function chaining (SFC) focuses mainly on deploying various network functions in geographically distributed data centers and providing interconnect routing among them. Traditional (convex optimization-based) SFC algorithms exhibit some disadvantages on the scalability and accuracy. Recently, researches have shown the effectiveness of deep reinforcement learning (DRL) in the field of SFC. However, current DRL-based algorithms possess an extremely large action space, which leads to slow convergence and poor scalability. Some researchers relieve this issue by reformulating the SFC problem, which usually results in low utilization and high cost. To address this issue, we develop a hybrid DRL-based framework which decouples the VNF deployment and flow routing into different modules. In the proposed framework, a DRL agent is only responsible for learning the policy of VNF deployment. We customize the structure of the agent base on deep deterministic policy gradient (DDPG) and adopt several techniques to improve the learning efficiency, such as adaptive parameter noise, wolpertinger policy, and prioritized experience replay. The flow routing is conducted in a game-based module (GBM). We design a decentralized routing algorithm for the GBM to address the scalability. The end-to-end latency of flows is minimized while the resource capacity and location constraints are satisfied. During the learning process of the proposed framework, the DRL agent improves its deployment policy with the reward from the GBM (the value of reward depends on flow routing). Thus, the VNF deployment and flow routing are still jointly optimized. Compared to existing DRL-learning algorithms, the proposed hybrid DRL framework can achieve a lower cost since 1) the action space is significantly reduced due to flow routing decoupling; 2) the flow routing procedure is more efficient (the GBM adopts model-based information, e.g., the gradient). Through trace-driven simulations, we show the efficiency of our algorithm compared to existing DRL-based algorithms.

Scalable Deep Reinforcement Learning-Based Online Routing for Multi-Type Service Requirements

Enabling Scalable Routing in Software-Defined Networks With Deep Reinforcement Learning on Critical Nodes

Improving the Scalability of Deep Reinforcement Learning-Based Routing with Control on Partial Nodes

Intelligent Routing Algorithm over SDN: Reusable Reinforcement Learning Approach

Deep Reinforcement Learning Based Dynamic Routing Optimization for Delay-Sensitive Applications

G-Routing: Graph Neural Networks-Based Flexible Online Routing

QoS Routing Optimization Based on Deep Reinforcement Learning in SDN

DRL-D: Revenue-Aware Online Service Function Chain Deployment Via Deep Reinforcement Learning

ERODRL: Efficient ISP Routing Optimization Via Deep Reinforcement Learning

A Deep Reinforcement Learning Approach for Adaptive Traffic Routing in Next-gen Networks

A Hybrid Deep Reinforcement Learning Routing Method under Dynamic and Complex Traffic with Software Defined Networking

DQS: A QoS-Driven Routing Optimization Approach in SDN using Deep Reinforcement Learning

DRL-QOR: Deep Reinforcement Learning-Based QoS/QoE-Aware Adaptive Online Orchestration in NFV-Enabled Networks

DRLAR: A Deep Reinforcement Learning-Based Adaptive Routing Framework for Network-on-chips

Toward Packet Routing With Fully Distributed Multiagent Deep Reinforcement Learning

Toward Packet Routing with Fully-distributed Multi-agent Deep Reinforcement Learning

An Intelligent SDWN Routing Algorithm Based on Network Situational Awareness and Deep Reinforcement Learning

Towards Real-Time Routing Optimization with Deep Reinforcement Learning: Open Challenges

A Graph reinforcement learning based SDN routing path selection for optimizing long-term revenue

A Hybrid Learning Framework for Service Function Chaining Across Geo-Distributed Data Centers.

Stable QoE-Aware Multi-SFCs Cooperative Routing Mechanism Based on Deep Reinforcement Learning