Abstract:Machine learning (ML) is widely being regarded as a powerful technology to solve various complex problems in fiberoptic communication networks. Recently, ML algorithms have been considered for a variety of use-cases in optical networks ranging from the design of network devices to the compensation of transmission impairments to predicting network-wide traffic flow patterns. As ML techniques are inherently data driven, the availability of relevant data sets largely dictates the performance of developed ML models. Presently, majority of ML-aided methods in the literature rely either on synthetic data from certain simulation tools or exploit data from some simple laboratory setups. However, developing robust ML-based solutions for real fiber-optic networks demands assembling big data from various parts as well as across different layers and domains of the network. Unfortunately, there is currently a lack of understanding among the research community about various crucial data aspects in practical optical networks as well as key challenges faced. While there has been a plethora of works discussing numerous ML applications in fiber-optic communications and networking, to the best of our knowledge, there is presently no paper which particularly focuses on data perspectives. In this paper, we aim to shed light on various critical data factors by first describing the tools and technologies imperative for adequate network monitoring and data acquisition. We also elucidate different data types, their physical sources, and their applications. Several useful data handling and pre-processing techniques for efficient network data collection, storage, presentation, analysis, etc., are discussed. Major challenges in the generation, transmission, and sharing of the data are highlighted and some potential solutions to the outstanding problems are proposed, thus paving the way for seamless availability of pertinent data sets essential for realizing intelligent data-driven operations in future fiber-optic networks.

AI/ML-as-a-Service for optical network automation: use cases and challenges

Machine Learning for Intelligent Optical Networks: A Comprehensive Survey

Artificial intelligence (AI) methods in optical networks: A comprehensive survey

Towards explainable artificial intelligence in optical networks: the use case of lightpath QoT estimation

An Overview on Application of Machine Learning Techniques in Optical Networks

Non-technological barriers: the last frontier towards AI-powered intelligent optical networks

AI-Based Modeling and Monitoring Techniques for Future Intelligent Elastic Optical Networks

Advances in Optical Communications and Network Technologies.

Two Decades of AI4NETS-AI/ML for Data Networks: Challenges & Research Directions

Coordination Between Control Layer AI and On-Board AI in Optical Transport Networks [Invited]

SOON: self-optimizing optical networks with machine learning.

Artificial Intelligence in Optical Communications: From Machine Learning to Deep Learning

Survey on Machine Learning for Traffic-Driven Service Provisioning in Optical Networks

Data perspectives in AI-assisted fiber-optic communication networks

When Large Language Models Meet Optical Networks: Paving the Way for Automation

Two Use Cases of Machine Learning for SDN-Enabled IP/Optical Networks: Traffic Matrix Prediction and Optical Path Performance Prediction

From Automation to Autonomous: Driving the Optical Network Management to Fixed Fifth-generation (F5G) Advanced

Study of Optical Networks, 5G, Artificial Intelligence and Their Applications

AI-Enabled Experience-Driven Networking: Vision, State-of-the-Art and Future Directions

Elevating optical networks: Machine learning approach for optimal resource scheduling and performance boost

Evolution towards Smart Optical Networking: Where Artificial Intelligence (AI) meets the World of Photonics