Machine Learning for Optical Layer Failure Management

Danshi Wang,Dongdong Wang,Chunyu Zhang,Lingling Wang,Songlin Liu,Min Zhang

DOI: https://doi.org/10.1364/oecc.2021.t3a.3

2021-01-01

Abstract:We review applications of machine learning (ML) in optical layer failure management, including alarm analysis, failure prediction, failure detection, failure diagnosis, and failure location driven by various algorithms from ML communities.

What problem does this paper attempt to address?

A Review of Machine Learning-based Failure Management in Optical Networks

Danshi Wang,Chunyu Zhang,Wenbin Chen,Hui Yang,Min Zhang,Alan Pak Tao Lau

DOI: https://doi.org/10.48550/arXiv.2208.10677

2022-08-23

Abstract:Failure management plays a significant role in optical networks. It ensures secure operation, mitigates potential risks, and executes proactive protection. Machine learning (ML) is considered to be an extremely powerful technique for performing comprehensive data analysis and complex network management and is widely utilized for failure management in optical networks to revolutionize the conventional manual methods. In this study, the background of failure management is introduced, where typical failure tasks, physical objects, ML algorithms, data source, and extracted information are illustrated in detail. An overview of the applications of ML in failure management is provided in terms of alarm analysis, failure prediction, failure detection, failure localization, and failure identification. Finally, the future directions on ML for failure management are discussed from the perspective of data, model, task, and emerging techniques.

Networking and Internet Architecture,Signal Processing
Automating Optical Network Fault Management with Machine Learning

Xiaoliang Chen,Che-Yu Liu,Roberto Proietti,Zhaohui Li,S. J. Ben Yoo

DOI: https://doi.org/10.1109/mcom.003.2200110

IF: 9.03

2022-12-14

IEEE Communications Magazine

Abstract:Effective fault management is essential for quality of service assurance in optical networks. Conventional fault management designs for optical networks mainly rely on threshold-based rules, which can hardly characterize the complex fault patterns therein. This article discusses the application of machine learning (ML) in actuating an automated optical network fault management architecture. The architecture is built on advanced optical performance monitoring (OPM) techniques and software-defined-networking-enabled programmable network control and management. With the viability of abundant OPM data, the architecture employs various ML models to automatically learn fault patterns and thereby realize data-driven cognitive fault management. We review the state of the art on fault detection, identification, and localization based on such an architecture, focusing specifically on soft failures. To overcome the applicability and scalability issues encountered by existing soft failure detection designs, we present a hybrid learning solution that combines the merits of supervised, unsupervised, and cooperative ML. In particular, we present a self-taught mechanism that self-learns fault patterns with unsupervised learning and then trains supervised learning classifiers with the learned patterns for online detection. We further introduce a broker-plane-aided federated learning framework to enable collaborative training of classifiers from multiple network domains while complying with domain privacy constraints. Performance evaluations show that the hybrid learning design can achieve high fault detection rates with negligible false alarms using less than 10 abnormal data samples (~ 0.1 percent of the scale of normal samples) for training.

telecommunications,engineering, electrical & electronic
Machine-Learning-Assisted Optical Fiber Communication System

Xinyu Zhou

DOI: https://doi.org/10.54097/hset.v27i.3826

2022-12-27

Abstract:With the development of software-defined networking and coherent transmission, to name only a couple of emerging technical and technological areas, optical networks have rapidly expanded during the past few years. To handle with the enormous increment, several sections of optical transmission networks have been addressed via machine learning. Techniques such as support vector machine and KNN algorithms are widely used in fiber-induced nonlinear mitigation, which can cause enormous financial loses if the problem is not solved properly. Optical performance monitoring is another essential area in optical networks, which often adopts multitasking, while failure management, where anomaly detection takes place, rely on both supervised and unsupervised learning methods. The overview includes a brief synopsis of four types of learning methods, including supervised learning, unsupervised learning, semi-supervised learning and reinforcement, as well as the most recent advancements in methodologies used in optical fibre communication. At the end of the analysis, it is demonstrated that ML algorithms are selected based on the challenges present and we have to consider multiple factors when choosing a suitable algorithm. The combination of the two fields brings prosperity to each other.

Engineering,Computer Science
Application of machine learning in optical fiber sensors

Yifan Zhou,Ya-nan Zhang,Qi Yu,Lirong Ren,Qi Liu,Yong Zhao

DOI: https://doi.org/10.1016/j.measurement.2024.114391

IF: 5.6

2024-03-01

Measurement

Abstract:In recent years, with the increasing demand for intelligent society, intelligent photonics has developed rapidly. Machine learning (ML), as a subset of artificial intelligence (AI), has played an important role in the intelligent evolution of optical fiber sensors. Its impact extends beyond enhancing sensor performance by introducing innovative problem-solving approaches. Specifically, ML algorithms have become instrumental in signal demodulation and elevating the efficacy of discrete and distributed sensors, and have also greatly promoted the development of optical fiber speckle pattern processing. This paper presents the latest advancements in ML-based optical fiber sensors, outlines the problems faced by conventional demodulation methods and the common ML algorithms applied in optical fiber sensors, and emphasizes key applications. Additionally, this paper delves into the challenges and future development of this emerging research direction.

engineering, multidisciplinary,instruments & instrumentation
Application of Machine Learning in Elastic Optical Networks.

Qunbi Zhuge,Weisheng Hu

DOI: https://doi.org/10.1109/ecoc.2018.8535565

2018-01-01

Abstract:Machine learning (ML) techniques have been adopted in elastic optical networks to enable automated operations and extract more capacity from the physical layer. We review and discuss various applications of ML in link monitoring, link modeling, link provisioning and network control.
Machine Learning Based Wafer Defect Detection

Yuansheng Ma,Feng Wang,Qian Xie,Le Hong,Joerg Mellmann,Yuyang Sun,Shao Wen Gao,Sonal Singh,Panneerselvam Venkatachalam,James Word

DOI: https://doi.org/10.1117/12.2513232

2019-01-01

Abstract:Detecting and resolving the true on-wafer-hotspot (defect) is critical to improve wafers' yield in high volume manufacturing semiconductor foundries. As the integrated circuits process becomes more and more complex with the technology scaling, Optical Rule Check (ORC) alone could no longer identify the outlier-alike defects i.e. hot yield killer defects. Failing to detect yield-killer defects could be due to the lack of sufficient understanding and modeling in terms of etching, CMP, as well as other inter-layer process variations. In this paper, we present a fast and accurate defect detection flow with machine learning (ML) methodologies to address the compounding effects from different process stages.There are three parts in the defect detection ML model building flow: the first part is on the feature generation and data collection, the second on the ML model building, and the third on the full-chip prediction. We use limited amount of known defects found on wafer as input to train the ML model, and then apply the ML model to the full chip for prediction. The wafer verification data showed that our flow achieved more than 80% of defect hit rate with engineered feature extractions and ML model for an advanced technology node mask. The wafer results showed that machine learning has the capabilities of identifying new types of defects patterns and high-risk repetitive patterns such as SRAM.
Machine Learning Applications for Short Reach Optical Communication

Yapeng Xie,Yitong Wang,Sithamparanathan Kandeepan,Ke Wang

DOI: https://doi.org/10.3390/photonics9010030

IF: 2.536

2022-01-04

Photonics

Abstract:With the rapid development of optical communication systems, more advanced techniques conventionally used in long-haul transmissions have gradually entered systems covering shorter distances below 100 km, where higher-speed connections are required in various applications, such as the optical access networks, inter- and intra-data center interconnects, mobile fronthaul, and in-building and indoor communications. One of the techniques that has attracted intensive interests in short-reach optical communications is machine learning (ML). Due to its robust problem-solving, decision-making, and pattern recognition capabilities, ML techniques have become an essential solution for many challenging aspects. In particular, taking advantage of their high accuracy, adaptability, and implementation efficiency, ML has been widely studied in short-reach optical communications for optical performance monitoring (OPM), modulation format identification (MFI), signal processing and in-building/indoor optical wireless communications. Compared with long-reach communications, the ML techniques used in short-reach communications have more stringent complexity and cost requirements, and also need to be more sensitive. In this paper, a comprehensive review of various ML methods and their applications in short-reach optical communications are presented and discussed, focusing on existing and potential advantages, limitations and prospective trends.

optics
Experimental Investigation of Machine Learning based Soft-Failure Management using the Optical Spectrum

Lars E. Kruse,Sebastian Kühl,Annika Dochhan,Stephan Pachnicke

DOI: https://doi.org/10.48550/arXiv.2312.07208

2023-12-12

Networking and Internet Architecture

Abstract:The demand for high-speed data is exponentially growing. To conquer this, optical networks underwent significant changes getting more complex and versatile. The increasing complexity necessitates the fault management to be more adaptive to enhance network assurance. In this paper, we experimentally compare the performance of soft-failure management of different machine learning algorithms. We further introduce a machine-learning based soft-failure management framework. It utilizes a variational autoencoder based generative adversarial network (VAE-GAN) running on optical spectral data obtained by optical spectrum analyzers. The framework is able to reliably run on a fraction of available training data as well as identifying unknown failure types. The investigations show, that the VAE-GAN outperforms the other machine learning algorithms when up to 10\% of the total training data is available in identification tasks. Furthermore, the advanced training mechanism for the GAN shows a high F1-score for unknown spectrum identification. The failure localization comparison shows the advantage of a low complexity neural network in combination with a VAE over established machine learning algorithms.
Machine Learning Based Optical Transmission System Link Performance Degradation Prediction and Application

ZhiQiang Fan,ZhenWei Wu,JianXin Lv,Peng Zhang,Yong Xiao

2023-01-01

Abstract:Optical networks play a critical role in modern information society. Statistics have revealed that around 50% of optical service faults stem from "errors" in the optical transmission system links, which are monitored through frame overheads. Unfortunately, current maintenance methods are insufficient in accurately predicting these errors, leading to potential disruptions in network services. To address this issue, this study introduces the Gini importance-based feature selection approach and maximum likelihood-based logistic regression algorithm for predicting optical transmission system link performance degradation. To facilitate the application of this model in real-world networks, the Network AI (NAI) three-level intelligent architecture is proposed, and an optical transmission performance degradation prediction system is developed based on this architecture. Experimental results from the network experiment demonstrate the effectiveness of the proposed performance degradation model and NAI-based system.
A Machine Learning-based Framework for Predictive Maintenance of Semiconductor Laser for Optical Communication

Khouloud Abdelli,Helmut Griesser,Stephan Pachnicke

DOI: https://doi.org/10.48550/arXiv.2211.02842

IF: 5.414

2022-11-05

Machine Learning

Abstract:Semiconductor lasers, one of the key components for optical communication systems, have been rapidly evolving to meet the requirements of next generation optical networks with respect to high speed, low power consumption, small form factor etc. However, these demands have brought severe challenges to the semiconductor laser reliability. Therefore, a great deal of attention has been devoted to improving it and thereby ensuring reliable transmission. In this paper, a predictive maintenance framework using machine learning techniques is proposed for real-time heath monitoring and prognosis of semiconductor laser and thus enhancing its reliability. The proposed approach is composed of three stages: i) real-time performance degradation prediction, ii) degradation detection, and iii) remaining useful life (RUL) prediction. First of all, an attention based gated recurrent unit (GRU) model is adopted for real-time prediction of performance degradation. Then, a convolutional autoencoder is used to detect the degradation or abnormal behavior of a laser, given the predicted degradation performance values. Once an abnormal state is detected, a RUL prediction model based on attention-based deep learning is utilized. Afterwards, the estimated RUL is input for decision making and maintenance planning. The proposed framework is validated using experimental data derived from accelerated aging tests conducted for semiconductor tunable lasers. The proposed approach achieves a very good degradation performance prediction capability with a small root mean square error (RMSE) of 0.01, a good anomaly detection accuracy of 94.24% and a better RUL estimation capability compared to the existing ML-based laser RUL prediction models.
Application of Machine Learning in Fiber Nonlinearity Modeling and Monitoring for Elastic Optical Networks

Qunbi Zhuge,Xiaobo Zeng,Huazhi Lun,Meng Cai,Xiaomin Liu,Lilin Yi,Weisheng Hu

DOI: https://doi.org/10.1109/jlt.2019.2910143

IF: 4.7

2019-01-01

Journal of Lightwave Technology

Abstract:Fiber nonlinear interference (NLI) modeling and monitoring are the key building blocks to support elastic optical networks. In the past, they were normally developed and investigated separately. Moreover, the accuracy of the previously proposed methods still needs to be improved for heterogenous dynamic optical networks. In this paper, we present the application of machine learning (ML) in NLI modeling and monitoring. In particular, we first propose to use ML approaches to calibrate the errors of current fiber nonlinearity models. The Gaussian-noise model is used as an illustrative example, and significant improvement is demonstrated with the aid of an artificial neural network. Further, we propose to use ML to combine the modeling and monitoring schemes for a better estimation of NLI variance. Extensive simulations with 2411 links are conducted to evaluate and analyze the performance of various schemes, and the superior performance of the ML-aided combination of modeling and monitoring is demonstrated.
Machine Learning for Optical Network Security Management

Marija Furdek,Carlos Natalino

DOI: https://doi.org/10.1364/ofc.2020.m4e.4

2020-01-01

Abstract:We discuss the role of supervised, unsupervised and semi-supervised learning techniques in identification of optical network security breaches. The applicability, performance and challenges related to practical deployment of these techniques are examined.
Machine Learning for Self-Coherent Detection Short-Reach Optical Communications

Qi Wu,Zhaopeng Xu,Yixiao Zhu,Yikun Zhang,Honglin Ji,Yu Yang,Gang Qiao,Lulu Liu,Shangcheng Wang,Junpeng Liang,Jinlong Wei,Jiali Li,Zhixue He,Qunbi Zhuge,Weisheng Hu

DOI: https://doi.org/10.3390/photonics10091001

IF: 2.536

2023-01-01

Photonics

Abstract:Driven by emerging technologies such as the Internet of Things, 4K/8K video applications, virtual reality, and the metaverse, global internet protocol traffic has experienced an explosive growth in recent years. The surge in traffic imposes higher requirements for the data rate, spectral efficiency, cost, and power consumption of optical transceivers in short-reach optical networks, including data-center interconnects, passive optical networks, and 5G front-haul networks. Recently, a number of self-coherent detection (SCD) systems have been proposed and gained considerable attention due to their spectral efficiency and low cost. Compared with coherent detection, the narrow-linewidth and high-stable local oscillator can be saved at the receiver, significantly reducing the hardware complexity and cost of optical modules. At the same time, machine learning (ML) algorithms have demonstrated a remarkable performance in various types of optical communication applications, including channel equalization, constellation optimization, and optical performance monitoring. ML can also find its place in SCD systems in these scenarios. In this paper, we provide a comprehensive review of the recent progress in SCD systems designed for high-speed optical short- to medium-reach transmission links. We discuss the diverse applications and the future perspectives of ML for these SCD systems.
An Overview on Application of Machine Learning Techniques in Optical Networks

Francesco Musumeci,Cristina Rottondi,Avishek Nag,Irene Macaluso,Darko Zibar,Marco Ruffini,Massimo Tornatore

DOI: https://doi.org/10.48550/arXiv.1803.07976

2018-12-01

Abstract:Today's telecommunication networks have become sources of enormous amounts of widely heterogeneous data. This information can be retrieved from network traffic traces, network alarms, signal quality indicators, users' behavioral data, etc. Advanced mathematical tools are required to extract meaningful information from these data and take decisions pertaining to the proper functioning of the networks from the network-generated data. Among these mathematical tools, Machine Learning (ML) is regarded as one of the most promising methodological approaches to perform network-data analysis and enable automated network self-configuration and fault management. The adoption of ML techniques in the field of optical communication networks is motivated by the unprecedented growth of network complexity faced by optical networks in the last few years. Such complexity increase is due to the introduction of a huge number of adjustable and interdependent system parameters (e.g., routing configurations, modulation format, symbol rate, coding schemes, etc.) that are enabled by the usage of coherent transmission/reception technologies, advanced digital signal processing and compensation of nonlinear effects in optical fiber propagation. In this paper we provide an overview of the application of ML to optical communications and networking. We classify and survey relevant literature dealing with the topic, and we also provide an introductory tutorial on ML for researchers and practitioners interested in this field. Although a good number of research papers have recently appeared, the application of ML to optical networks is still in its infancy: to stimulate further work in this area, we conclude the paper proposing new possible research directions.

Networking and Internet Architecture,Machine Learning
Machine Learning in Short-Reach Optical Systems: A Comprehensive Survey

Chen Shao,Elias Giacoumidis,Syed Moktacim Billah,Shi Li,Jialei Li,Prashasti Sahu,Andre Richter,Tobias Kaefer,Michael Faerber

2024-05-29

Abstract:In recent years, extensive research has been conducted to explore the utilization of machine learning algorithms in various direct-detected and self-coherent short-reach communication applications. These applications encompass a wide range of tasks, including bandwidth request prediction, signal quality monitoring, fault detection, traffic prediction, and digital signal processing (DSP)-based equalization. As a versatile approach, machine learning demonstrates the ability to address stochastic phenomena in optical systems networks where deterministic methods may fall short. However, when it comes to DSP equalization algorithms, their performance improvements are often marginal, and their complexity is prohibitively high, especially in cost-sensitive short-reach communications scenarios such as passive optical networks (PONs). They excel in capturing temporal dependencies, handling irregular or nonlinear patterns effectively, and accommodating variable time intervals. Within this extensive survey, we outline the application of machine learning techniques in short-reach communications, specifically emphasizing their utilization in high-bandwidth demanding PONs. Notably, we introduce a novel taxonomy for time-series methods employed in machine learning signal processing, providing a structured classification framework. Our taxonomy categorizes current time series methods into four distinct groups: traditional methods, Fourier convolution-based methods, transformer-based models, and time-series convolutional networks. Finally, we highlight prospective research directions within this rapidly evolving field and outline specific solutions to mitigate the complexity associated with hardware implementations. We aim to pave the way for more practical and efficient deployment of machine learning approaches in short-reach optical communication systems by addressing complexity concerns.

Signal Processing,Machine Learning
Applying machine learning to optical metrology: a review

Ruidong Xue,Helia Hooshmand,Mohammed Isa,Samanta Piano,Richard K Leach

DOI: https://doi.org/10.1088/1361-6501/ad7878

IF: 2.398

2024-09-11

Measurement Science and Technology

Abstract:This literature review investigates the integration of machine learning (ML) into optical metrology, unveiling enhancements in both efficiency and effectiveness of measurement processes. With a focus on phase demodulation, unwrapping, and phase-to-height conversion, the review highlights how ML algorithms have transformed traditional optical metrology techniques, offering improved speed, accuracy, and data processing capabilities. Efficiency improvements are underscored by advancements in data generation, intelligent sampling, and processing strategies, where ML algorithms have accelerated the metrological evaluations. Effectiveness is enhanced in measurement precision, with ML providing robust solutions to complex pattern recognition and noise reduction challenges. Additionally, the role of parallel computing using graphics processing units (GPUs) and field programmable gate arrays (FPGAs) is emphasised, showcasing their importance in supporting the computationally intensive ML algorithms for real-time processing. This review culminates in identifying future research directions, emphasising the potential of advanced ML models and broader applications within optical metrology. Through this investigation, the review articulates a future where optical metrology, empowered by ML, achieves improved levels of operational efficiency and effectiveness.

engineering, multidisciplinary,instruments & instrumentation
Forecasting loss of signal in optical networks with machine learning

Wenjie Du,David Cote,Chris Barber,Yan Liu,David Côté

DOI: https://doi.org/10.1364/jocn.423667

IF: 4.508

2021-07-10

Journal of Optical Communications and Networking

Abstract:Wenjie Du, David Côté, Chris Barber, Yan Liu Loss of signal (LOS) represents a significant cost for operators of optical networks. By studying large sets of real-world performance ... [J. Opt. Commun. Netw. 13, E109-E121 (2021)]

computer science, information systems,telecommunications, hardware & architecture,optics
Machine Learning for Intelligent Optical Networks: A Comprehensive Survey

Rentao Gu,Zeyuan Yang,Yuefeng Ji

DOI: https://doi.org/10.1016/j.jnca.2020.102576

2020-03-11

Abstract:With the rapid development of Internet and communication systems, both in services and technologies, communication networks have been suffering increasing complexity. It is imperative to improve intelligence in communication network, and several aspects have been incorporating with Artificial Intelligence (AI) and Machine Learning (ML). Optical network, which plays an important role both in core and access network in communication networks, also faces great challenges of system complexity and the requirement of manual operations. To overcome the current limitations and address the issues of future optical networks, it is essential to deploy more intelligence capability to enable autonomous and exible network operations. ML techniques are proved to have superiority on solving complex problems; and thus recently, ML techniques have been used for many optical network applications. In this paper, a detailed survey of existing applications of ML for intelligent optical networks is presented. The applications of ML are classified in terms of their use cases, which are categorized into optical network control and resource management, and optical networks monitoring and survivability. The use cases are analyzed and compared according to the used ML techniques. Besides, a tutorial for ML applications is provided from the aspects of the introduction of common ML algorithms, paradigms of ML, and motivations of applying ML. Lastly, challenges and possible solutions of ML application in optical networks are also discussed, which intends to inspire future innovations in leveraging ML to build intelligent optical networks.

Networking and Internet Architecture,Machine Learning
An Overview of ML-based Applications for Next Generation Optical Networks

Ruoxuan Gao,Lei Liu,Xiaomin Liu,Huazhi Lun,Lilin Yi,Weisheng Hu,Qunbi Zhuge

DOI: https://doi.org/10.1007/s11432-020-2874-y

2020-01-01

Science China Information Sciences

Abstract:Over the past few decades, the demand for the capacity and reliability of optical networks has continued to grow. In the meantime, optical networks with larger knowledge scales have become sources of numerous heterogeneous data. In order to handle these new challenges, many issues need to be resolved,among which the low-margin optical networks design, power optimization, routing and wavelength assignment(RWA), failure management are quite important. However, the use of traditional algorithms in the above four applications shows some shortcomings. Fortunately, artificial intelligence(AI), especially machine learning(ML), is regarded as one of the most promising methods to overcome these shortcomings. In this study, we review the applications of ML methods in solving these four issues. Although many ML-based researches have emerged, the applications of ML techniques in optical networks still face challenges. Therefore, we also discuss some possible future directions of investigating ML-based approaches in optical networks.
Fault Monitoring in Passive Optical Networks using Machine Learning Techniques

Khouloud Abdelli,Carsten Tropschug,Helmut Griesser,Stephan Pachnicke

2023-07-08

Abstract:Passive optical network (PON) systems are vulnerable to a variety of failures, including fiber cuts and optical network unit (ONU) transmitter/receiver failures. Any service interruption caused by a fiber cut can result in huge financial losses for service providers or operators. Identifying the faulty ONU becomes difficult in the case of nearly equidistant branch terminations because the reflections from the branches overlap, making it difficult to distinguish the faulty branch given the global backscattering signal. With increasing network size, the complexity of fault monitoring in PON systems increases, resulting in less reliable monitoring. To address these challenges, we propose in this paper various machine learning (ML) approaches for fault monitoring in PON systems, and we validate them using experimental optical time domain reflectometry (OTDR) data.

Machine Learning,Networking and Internet Architecture,Signal Processing

Machine Learning for Optical Layer Failure Management

A Review of Machine Learning-based Failure Management in Optical Networks

Automating Optical Network Fault Management with Machine Learning

Machine-Learning-Assisted Optical Fiber Communication System

Application of machine learning in optical fiber sensors

Application of Machine Learning in Elastic Optical Networks.

Machine Learning Based Wafer Defect Detection

Machine Learning Applications for Short Reach Optical Communication

Experimental Investigation of Machine Learning based Soft-Failure Management using the Optical Spectrum

Machine Learning Based Optical Transmission System Link Performance Degradation Prediction and Application

A Machine Learning-based Framework for Predictive Maintenance of Semiconductor Laser for Optical Communication

Application of Machine Learning in Fiber Nonlinearity Modeling and Monitoring for Elastic Optical Networks

Machine Learning for Optical Network Security Management

Machine Learning for Self-Coherent Detection Short-Reach Optical Communications

An Overview on Application of Machine Learning Techniques in Optical Networks

Machine Learning in Short-Reach Optical Systems: A Comprehensive Survey

Applying machine learning to optical metrology: a review

Forecasting loss of signal in optical networks with machine learning

Machine Learning for Intelligent Optical Networks: A Comprehensive Survey

An Overview of ML-based Applications for Next Generation Optical Networks

Fault Monitoring in Passive Optical Networks using Machine Learning Techniques