Abstract:The rapid growth of online services and the global transition to digital platforms, accelerated by the COVID-19 pandemic and the emergence of AI-based services, have led to a surge in demand for high-capacity optical communication networks (OCNs). However, this transition exacerbates the issue of fiber nonlinearity, notably four-wave mixing (FWM), which can significantly impact signal integrity in long-distance OCN transmissions with multiple channels. To address this challenge, this paper presents a novel approach leveraging a convolutional neural network (CNN)-based digital signal processing (DSP) model to mitigate the nonlinear effects of FWM on signal quality. The proposed CNN model, designed with three convolutional layers and trained on a comprehensive dataset of simulated OCN transmissions, demonstrates significant improvement in reducing nonlinear distortions caused by FWM. Our method is compared with traditional compensation techniques, such as digital back-propagation and Volterra series-based equalizers, showing a reduction in power penalties by 1 dB and an enhancement in the power budget by 2 dB. Simulation results indicate that the bit error rates (BERs) remain consistently below the critical threshold of 10 −9 across various transmission distances, outperforming conventional methods. The simulation analysis, conducted on a 100 km multi-channel OCN testbed, confirms the CNN equalizer's efficacy in reducing crosstalk and improving power efficiency, with a demonstrated 15% reduction in required launch power. Moreover, the analytical models used to evaluate the CNN-based DSP model show high accuracy in predicting performance, underscoring the model's capability to achieve high-quality transmission with lower power requirements. These findings advocate for the integration of our CNN-based DSP model into OCNs, offering a fruitful solution for managing multi-channel, long-distance, and high-data-rate optical transmissions in real-world systems.

CNN-Based Inter-Band Interference Cancellation for 100Gbit/s/λ Non-Orthogonal m-CAP in Fronthaul

Inter-Band Interference Cancellation Based on Complex ICA for 100gbit/s/λ Non-Orthogonal M-Cap NGFI-II Fronthaul Data Transmission

Inter-Band Interference Cancellation Based on Independent Component Analysis for 200gb/s/λ Non-Orthogonal M-Cap NGFI-II Data Transmission in 5G Fronthaul.

Network assisted inter-cell codeword cancellation for interference-limited LTE-A and beyond

Hybrid Fiber-Thz Fronthaul Supporting Up to 16384-QAM-OFDM with the Delta-Sigma Modulation

Interference Mitigation for Faster-Than-Nyquist Multiband CAP Visible Light Communication

Mitigation of Multipath Interference Utilizing High-Pass Feed-Forward Equalization and 1-Dimension Convolutional Neural Network

A Hybrid Network Integrating MHSA and 1D CNN–Bi-LSTM for Interference Mitigation in Faster-than-Nyquist MIMO Optical Wireless Communications

High Spectrum Compression Scheme Based on DNN Post Equalization for Multiband Quadrature Dual‐quaternary Coded FTN CAP 49QAM Signals in Visible Light Communication System

Complex principal component analysis-based complex-valued fully connected NN equalizer for optical fibre communications

Improving Optical Transmission Performance in Multi-Channel Networks Utilizing Convolutional Neural Network-Enabled Digital Signal Processing to Mitigate Four-Wave Mixing

High-Capacity Tier-II Fronthaul Network with SSB-DD Multiband OQAM/QAM-CAP

Demonstration of photonic neural network for fiber nonlinearity compensation in long-haul transmission systems

Convolutional Neural Network-Aided DP-64 QAM Coherent Optical Communication Systems

Cell-free ISAC Massive MIMO Systems with Capacity-Constrained Fronthaul Links

Neural-network-based direct waveform to symbol conversion for joint ISI and ICI cancellation in non-orthogonal multi-band CAP based UDWDM fiber-mmWave integration

Enhanced Performance Utilizing Joint Processing Algorithm for CAP Signals

Faster-Than-Nyquist N-Dimensional CAP Modulation Scheme Based on Slicing Sphere Decoder for Visible Light Communication

Global Receptive Field Designed Complex-Valued Convolutional Neural Network Equalizer for Optical Fiber Communications

A 0.90-Tb/s/in 1.29-Pj/b Wireline Transceiver with Single-Ended Crosstalk Cancellation Coding Scheme for High-Density Interconnects.

CNN-Based Equalization for Communications: Achieving Gigabit Throughput with a Flexible FPGA Hardware Architecture