Abstract:The service provisioning in multi-domain software-defined elastic optical networks (SD-EONs) is an interesting but difficult problem to tackle, because the basic problem of lightpath provisioning, i.e., the routing and spectrum assignment (RSA), is $\mathcal {NP}$ -hard, and each domain is owned and operated by a different carrier. Therefore, even though numerous RSA heuristics have been proposed, there does not exist a universal winner that can always achieve the lowest blocking probability in all the scenarios of a multi-domain SD-EON. This motivates us to revisit the inter-domain provisioning problem in this paper by leveraging deep reinforcement learning (DRL). Specifically, we propose DeepCoop, which is an inter-domain service framework that uses multiple cooperative DRL agents to achieve scalable network automation in a multi-domain SD-EON. DeepCoop employs a DRL agent in each domain to optimize intra-domain service provisioning, while a domain-level path computation element (PCE) is introduced to obtain the sequence of the domains to go through for each lightpath request. By sharing a restricted amount of information among each other, the DRL agents can make their decisions distributedly. To ensure scalability and universality, we design the action space of each DRL agent based on well-known RSA heuristics, and architect the agents based on the soft actor-critic (SAC) scenario. We run extensive simulations to evaluate DeepCoop, and the results show that DeepCoop can adapt to the dynamic environment in a multi-domain SD-EON to always select the best RSA heuristic for minimizing blocking probability, and it outperforms the existing algorithms on inter-domain provisioning in various scenarios. Moreover, we verify that the distributed training implemented in DeepCoop ensures its universality and scalability (i.e., its training and operation do not depend on the topology of the SD-EON).

Multi-Agent Deep Reinforcement Learning in Cognitive Inter-Domain Networking with Multi-Broker Orchestration

Multi-Agent and Cooperative Deep Reinforcement Learning for Scalable Network Automation in Multi-Domain SD-EONs.

Hierarchical Learning for Cognitive End-to-End Service Provisioning in Multi-Domain Autonomous Optical Networks

Multi-broker based software-defined optical networks (Invited paper)

Leveraging Deep Learning To Achieve Knowledge-Based Autonomous Service Provisioning In Broker-Based Multi-Domain Sd-Eons With Proactive And Intelligent Predictions Of Multi-Domain Traffic

Broker-Based Multi-Task Gaming to Facilitate Profit-Driven Network Orchestration in Multi-Domain Sd-Eons

DeepCoop: Leveraging Cooperative DRL Agents to Achieve Scalable Network Automation for Multi-Domain SD-EONs

Demonstration of Distributed Collaborative Learning with End-to-end QoT Estimation in Multi-Domain Elastic Optical Networks

Experimental Demonstration of Brokered Orchestration for End-to-end Service Provisioning and Interoperability Across Heterogeneous Multi-Operator (Multi-As) Optical Networks

Resource Allocation in Multicore Elastic Optical Networks: A Deep Reinforcement Learning Approach

A Multi-Task-Learning-Based Transfer Deep Reinforcement Learning Design for Autonomic Optical Networks

Multi-Agent DRL for Distributed Routing and Data Scheduling in Interplanetary Networks

Multi-Tenant Cross-Slice Resource Orchestration: A Deep Reinforcement Learning Approach

Joint Traffic Control and Multi-Channel Reassignment for Core Backbone Network in SDN-IoT: A Multi-Agent Deep Reinforcement Learning Approach

Reconfiguring Multicast Sessions in Elastic Optical Networks Adaptively with Graph-Aware Deep Reinforcement Learning

Multi-broker Based Market-Driven Service Provisioning in Multi-Domain SD-EONs in Nuncooperative Game Scenarios.

Exploiting Multi-Task Learning to Achieve Effective Transfer Deep Reinforcement Learning in Elastic Optical Networks.

MA-CDMR: An Intelligent Cross-domain Multicast Routing Method based on Multiagent Deep Reinforcement Learning in Multi-domain SDWN

Automatically Reconfigurable Optical Network for HPC System Based on Deep Reinforcement Learning

Incentive-Driven Bidding Strategy for Brokers to Compete for Service Provisioning Tasks in Multi-Domain SD-EONs

Multi-Task Multi-Agent Shared Layers are Universal Cognition of Multi-Agent Coordination