Abstract:Service function chaining (SFC) focuses mainly on deploying various network functions in geographically distributed data centers and providing interconnect routing among them. Traditional (convex optimization-based) SFC algorithms exhibit some disadvantages on the scalability and accuracy. Recently, researches have shown the effectiveness of deep reinforcement learning (DRL) in the field of SFC. However, current DRL-based algorithms possess an extremely large action space, which leads to slow convergence and poor scalability. Some researchers relieve this issue by reformulating the SFC problem, which usually results in low utilization and high cost. To address this issue, we develop a hybrid DRL-based framework which decouples the VNF deployment and flow routing into different modules. In the proposed framework, a DRL agent is only responsible for learning the policy of VNF deployment. We customize the structure of the agent base on deep deterministic policy gradient (DDPG) and adopt several techniques to improve the learning efficiency, such as adaptive parameter noise, wolpertinger policy, and prioritized experience replay. The flow routing is conducted in a game-based module (GBM). We design a decentralized routing algorithm for the GBM to address the scalability. The end-to-end latency of flows is minimized while the resource capacity and location constraints are satisfied. During the learning process of the proposed framework, the DRL agent improves its deployment policy with the reward from the GBM (the value of reward depends on flow routing). Thus, the VNF deployment and flow routing are still jointly optimized. Compared to existing DRL-learning algorithms, the proposed hybrid DRL framework can achieve a lower cost since 1) the action space is significantly reduced due to flow routing decoupling; 2) the flow routing procedure is more efficient (the GBM adopts model-based information, e.g., the gradient). Through trace-driven simulations, we show the efficiency of our algorithm compared to existing DRL-based algorithms.

GSC: Generalizable Service Coordination

GSC: A Graph-Based Skill Composition Framework for Robot Learning

Scalable Orchestration of Service Function Chains in NFV-Enabled Networks: A Federated Reinforcement Learning Approach

Efficient Task Migration and Resource Allocation in Cloud–edge Collaboration: A DRL Approach with Learnable Masking

Graph Neural Network Aided Deep Reinforcement Learning for Microservice Deployment in Cooperative Edge Computing

Service Coordination in the Space-Air-Ground Integrated Network

A Hybrid Learning Framework for Service Function Chaining Across Geo-Distributed Data Centers.

Combining graph neural network with deep reinforcement learning for resource allocation in computing force networks

DeepScaler: Holistic Autoscaling for Microservices Based on Spatiotemporal GNN with Adaptive Graph Learning

A Deep RL Approach on Task Placement and Scaling of Edge Resources for Cellular Vehicle-to-Network Service Provisioning

Intelligent Joint Network Slicing and Routing via GCN-powered Multi-Task Deep Reinforcement Learning

Group-Aware Coordination Graph for Multi-Agent Reinforcement Learning

Cognitive Coordination of Global Service Delivery

RLCS: Towards a robust and efficient mobile edge computing resource scheduling and task offloading system based on graph neural network

Graph Neural Network-based Multi-agent Reinforcement Learning for Resilient Distributed Coordination of Multi-Robot Systems

Seek Common While Shelving Differences: Orchestrating Deep Neural Networks for Edge Service Provisioning

GA-DRL: Graph Neural Network-Augmented Deep Reinforcement Learning for DAG Task Scheduling over Dynamic Vehicular Clouds

Edge-edge Collaboration Based Micro-service Deployment in Edge Computing Networks

Offloading Strategy Based on Graph Neural Reinforcement Learning in Mobile Edge Computing

A Multi-Agent Reinforcement Learning Scheme for SFC Placement in Edge Computing Networks

Scalable and Transferable Reinforcement Learning for Multi-Agent Mixed Cooperative–Competitive Environments Based on Hierarchical Graph Attention