Abstract:The service provisioning in multi-domain software-defined elastic optical networks (SD-EONs) is an interesting but difficult problem to tackle, because the basic problem of lightpath provisioning, i.e., the routing and spectrum assignment (RSA), is $\mathcal {NP}$ -hard, and each domain is owned and operated by a different carrier. Therefore, even though numerous RSA heuristics have been proposed, there does not exist a universal winner that can always achieve the lowest blocking probability in all the scenarios of a multi-domain SD-EON. This motivates us to revisit the inter-domain provisioning problem in this paper by leveraging deep reinforcement learning (DRL). Specifically, we propose DeepCoop, which is an inter-domain service framework that uses multiple cooperative DRL agents to achieve scalable network automation in a multi-domain SD-EON. DeepCoop employs a DRL agent in each domain to optimize intra-domain service provisioning, while a domain-level path computation element (PCE) is introduced to obtain the sequence of the domains to go through for each lightpath request. By sharing a restricted amount of information among each other, the DRL agents can make their decisions distributedly. To ensure scalability and universality, we design the action space of each DRL agent based on well-known RSA heuristics, and architect the agents based on the soft actor-critic (SAC) scenario. We run extensive simulations to evaluate DeepCoop, and the results show that DeepCoop can adapt to the dynamic environment in a multi-domain SD-EON to always select the best RSA heuristic for minimizing blocking probability, and it outperforms the existing algorithms on inter-domain provisioning in various scenarios. Moreover, we verify that the distributed training implemented in DeepCoop ensures its universality and scalability (i.e., its training and operation do not depend on the topology of the SD-EON).

Deep-RMSA: A Deep-Reinforcement-Learning Routing, Modulation and Spectrum Assignment Agent for Elastic Optical Networks

DeepRMSA: A Deep Reinforcement Learning Framework for Routing, Modulation and Spectrum Assignment in Elastic Optical Networks

Deep Reinforcement Learning-Based Routing and Spectrum Assignment of EONs by Exploiting GCN and RNN for Feature Extraction

Resource Allocation in Multicore Elastic Optical Networks: A Deep Reinforcement Learning Approach

Multi-Agent and Cooperative Deep Reinforcement Learning for Scalable Network Automation in Multi-Domain SD-EONs.

Temporal and Spectral 2D Fragmentation-Aware RMSA Algorithm for Advance Reservation Requests in EONs

Enhancing Routing in SD-EONs through Reinforcement Learning: A Comparative Analysis

Scalable Deep Reinforcement Learning for Routing and Spectrum Access in Physical Layer

A Multi-Task-Learning-Based Transfer Deep Reinforcement Learning Design for Autonomic Optical Networks

Exploiting Multi-Task Learning to Achieve Effective Transfer Deep Reinforcement Learning in Elastic Optical Networks.

On Deep Reinforcement Learning for Static Routing and Wavelength Assignment

Dynamic RMSA in spectrum-sliced elastic optical networks for high-throughput service provisioning

Dynamic RMSA in Elastic Optical Networks with an Adaptive Genetic Algorithm.

A Routing Optimization Method for Software-Defined Optical Transport Networks Based on Ensembles and Reinforcement Learning

Reconfiguring Multicast Sessions in EONs Adaptively with Deep Reinforcement Learning: (Invited Paper)

Routing, Modulation Level, Spectrum and Transceiver Assignment in Elastic Optical Networks.

Reconfiguring Multicast Sessions in Elastic Optical Networks Adaptively with Graph-Aware Deep Reinforcement Learning

Exact solution of the full RMSA problem in elastic optical networks

Energy Efficiency Optimization for Subterranean LoRaWAN Using A Reinforcement Learning Approach: A Direct-to-Satellite Scenario

Multi-Agent Deep Reinforcement Learning in Cognitive Inter-Domain Networking with Multi-Broker Orchestration

Multi-Agent Deep Reinforcement Learning for Distributed Satellite Routing