Abstract:Deep reinforcement learning (DRL) enables autonomic optical networking by allowing the network control and management systems to self-learn successful networking policies from operational experiences. This paper proposes a transfer learning approach for effective and scalable DRL in optical networks. We first present a modular DRL agent design to facilitate retrieving and transferring relevant knowledge between tasks requiring different dimensions of network state data. In particular, we partition network state data into common states, which contain generic information critical to multiple tasks (e.g., the spectrum utilization on fiber links), and task-specific states that are only used by a specific task (e.g., the utilization of virtual network functions). Separate neural network blocks are employed to process different state data. Based on the modular agent design, a multi-task learning (MTL) aided knowledge transferring scheme is proposed. The proposed scheme trains an MTL agent that can master multiple tasks simultaneously and thus enables to learn and transfer better-generalized knowledge across tasks. We perform case studies on the proposed transfer DRL approach considering two scenarios, namely, knowledge transferring between routing, modulation and spectrum assignment (RMSA) tasks for different networks and knowledge transferring from RMSA tasks to anycast service provisioning tasks. The DRL designs for RMSA and anycast service provisioning, including the state, action, and reward formulations and the training mechanisms, are also elaborated. Performance evaluations under both scenarios show that the proposed approach can effectively expedite the training processes of the target tasks and improve the ultimate service throughput.

Demonstration of Image Processing Based on Reinforcement Learning in Multi-Modal Optical Transport Networks

Cost-efficient Routing, Modulation, Wavelength and Port Assignment Using Reinforcement Learning in Optical Transport Networks

Reinforcement-Learning-Based Multi-Failure Restoration In Optical Transport Networks

Multi-Objective Routing and Resource Allocation Based on Reinforcement Learning in Optical Transport Networks

Service Restoration in Multi-Modal Optical Transport Networks with Reinforcement Learning.

A Routing Optimization Method for Software-Defined Optical Transport Networks Based on Ensembles and Reinforcement Learning

The First Testbed Demonstration Of Cognitive End-To-End Optical Service Provisioning With Hierarchical Learning Across Multiple Autonomous Systems

ADMIRE: Demonstration of Collaborative Data-Driven and Model-Driven Intelligent Routing Engine for IP/Optical Cross-Layer Optimization in X-Haul Networks

Demonstration of Distributed Collaborative Learning with End-to-end QoT Estimation in Multi-Domain Elastic Optical Networks

Experimental Demonstration of Machine-Learning-aided QoT Estimation in Multi-Domain Elastic Optical Networks with Alien Wavelengths

Hierarchical Learning for Cognitive End-to-End Service Provisioning in Multi-Domain Autonomous Optical Networks

Reinforced Resource Allocation Based on N-Dimensional Matrix Diagram for Multi-Modal Optical Networks

Actor-Critic-Based Resource Allocation for Multi-Modal Optical Networks

Demonstration of Parallel Service Re-Provisioning over Advanced Reservation Enabled Software Defined Optical Transport Networks

Automatically Reconfigurable Optical Network for HPC System Based on Deep Reinforcement Learning

Demonstration of Joint Optimization between Cloud AI and On-board AI in Optical Transport Networks

DNN Distributed Inference Offloading Scheme Based on Transfer Reinforcement Learning in Metro Optical Networks

Reconfigurable Optical Datacom Networks by Self-supervised Learning.

A Multi-Task-Learning-Based Transfer Deep Reinforcement Learning Design for Autonomic Optical Networks

Opti-DeepRoute: A Topology-Adaptive Deep Reinforcement Learning Based Service Provisioning Framework for Elastic Optical Network

Reinforcement-Learning based routing for packet-optical networks with hybrid telemetry