Abstract:The service provisioning in multi-domain software-defined elastic optical networks (SD-EONs) is an interesting but difficult problem to tackle, because the basic problem of lightpath provisioning, i.e., the routing and spectrum assignment (RSA), is $\mathcal {NP}$ -hard, and each domain is owned and operated by a different carrier. Therefore, even though numerous RSA heuristics have been proposed, there does not exist a universal winner that can always achieve the lowest blocking probability in all the scenarios of a multi-domain SD-EON. This motivates us to revisit the inter-domain provisioning problem in this paper by leveraging deep reinforcement learning (DRL). Specifically, we propose DeepCoop, which is an inter-domain service framework that uses multiple cooperative DRL agents to achieve scalable network automation in a multi-domain SD-EON. DeepCoop employs a DRL agent in each domain to optimize intra-domain service provisioning, while a domain-level path computation element (PCE) is introduced to obtain the sequence of the domains to go through for each lightpath request. By sharing a restricted amount of information among each other, the DRL agents can make their decisions distributedly. To ensure scalability and universality, we design the action space of each DRL agent based on well-known RSA heuristics, and architect the agents based on the soft actor-critic (SAC) scenario. We run extensive simulations to evaluate DeepCoop, and the results show that DeepCoop can adapt to the dynamic environment in a multi-domain SD-EON to always select the best RSA heuristic for minimizing blocking probability, and it outperforms the existing algorithms on inter-domain provisioning in various scenarios. Moreover, we verify that the distributed training implemented in DeepCoop ensures its universality and scalability (i.e., its training and operation do not depend on the topology of the SD-EON).

Hierarchical Learning for Cognitive End-to-End Service Provisioning in Multi-Domain Autonomous Optical Networks

Demonstration of Distributed Collaborative Learning with End-to-end QoT Estimation in Multi-Domain Elastic Optical Networks

The First Testbed Demonstration Of Cognitive End-To-End Optical Service Provisioning With Hierarchical Learning Across Multiple Autonomous Systems

Multi-Agent and Cooperative Deep Reinforcement Learning for Scalable Network Automation in Multi-Domain SD-EONs.

Multi-Agent Deep Reinforcement Learning in Cognitive Inter-Domain Networking with Multi-Broker Orchestration

Leveraging Deep Learning To Achieve Knowledge-Based Autonomous Service Provisioning In Broker-Based Multi-Domain Sd-Eons With Proactive And Intelligent Predictions Of Multi-Domain Traffic

Service Provisioning in Multi-Domain SD-EONs

Service Provisioning with Energy-Aware Regenerator Allocation in Multi-Domain EONs

Experimental Demonstration of Cognitive Provisioning and Alien Wavelength Monitoring in Multi-domain EON

Multibroker-Based Service Provisioning in Multidomain SD-EONs: Why and How Should the Brokers Cooperate With Each Other?

Experimental Demonstration of Machine-Learning-aided QoT Estimation in Multi-Domain Elastic Optical Networks with Alien Wavelengths

Intelligent Inter-Domain Connection Provisioning for Multi-Domain Multi-Vendor Optical Networks

DeepCoop: Leveraging Cooperative DRL Agents to Achieve Scalable Network Automation for Multi-Domain SD-EONs

Incentive-Driven Bidding Strategy for Brokers to Compete for Service Provisioning Tasks in Multi-Domain SD-EONs

DL-Assisted Cross-Layer Orchestration in Software-Defined IP-Over-EONs: from Algorithm Design to System Prototype

Experimental Demonstration of Brokered Orchestration for End-to-end Service Provisioning and Interoperability Across Heterogeneous Multi-Operator (Multi-As) Optical Networks

Demonstration of Cooperative Resource Allocation in an OpenFlow-Controlled Multidomain and Multinational SD-EON Testbed

Attack-Aware Service Provisioning to Enhance Physical-Layer Security in Multi-Domain EONs

Economics of Multi-Domain Software-Defined EONs - Games Among Brokers.

To Cooperate or Not to Cooperate: Service Function Chaining in Multi-Domain Edge-Cloud Elastic Optical Networks

Multi-broker Based Market-Driven Service Provisioning in Multi-Domain SD-EONs in Nuncooperative Game Scenarios.