Abstract:Recently, end-to-end deep learning (E2EDL) has been proposed for communication systems to improve the overall performance of systems. In the domain of optical fiber communication systems, the theory of E2EDL and the backpropagation (BP) blocking problem have not been reported in detail. Many works focus on the simple channel with an explicit model (e.g., self-phase modulation-only or dispersion-only fiber channel), while long-haul fiber transmission with both dispersion and nonlinearity has not been discussed because of the BP blocking problem. This paper analyzes why end-to-end deep learning can optimize the transmitter effectively in theory and derives the mean square error (MSE) loss for E2EDL, which is suitable for practice training. We discuss the BP problem mathematically and utilize a gradient estimation method called differentiable surrogate channel (DSC) to address the BP blocking problem. For the first time, the E2EDL is applied to a long-haul fiber channel to optimize the transmitter by using DSC-based gradient estimation. The simulation results demonstrate the mutual information (MI) and the generalized mutual information (GMI) increment with the training process, verifying the effectiveness of the E2EDL in long-haul fiber transmission. In addition, the proposed algorithm is applied to geometric constellations shaping (GCS) for long-haul transmission to explore the performance boundary of GCS. After GCS-based optical fiber E2EDL, the Q factor and the GMI can be improved around 0.25 dB and 0.11 bitssym at 1200 km for 48 GBaud 64-ary dual-polarization signal compared with the square QAM signal, respectively. In addition, the trained 16-ary, 32-ary, and 64-ary GCSs are tested from 800 km to 2800 km, 800 km to 2000 km, and 400 km to 1200 km, respectively. We believe the proposed E2EDL with DSC can achieve a tight lower bound of GCS performance limitation and open the door to transmitter optimization in the practical long-haul fiber transmission.

End-to-End Joint Digital and Optical Signal Processing Enabled by Interpretable Deep Learning for Coherent Optical Communication Systems

Wide and Deep Learning-Aided Nonlinear Equalizer for Coherent Optical Communication Systems

Optical Amplification-Free Deep Reservoir Computing-Assisted High-Baudrate Short-Reach Communication

End-to-End Deep Learning of Optical Fiber Communications

End-to-End Optimization and Equalization Based on Deep-Learning for Fiber-Terahertz Integrated Communication System at 209 GHz

An Optoelectronic Joint Equalization Scheme Based on Photonic Reservoir Computing and TDNN for CD and Nonlinearity Compensation in IM/DD Link

Low-Resolution Joint Encoding-Modulation Optical Fiber Communication System

Advancing theoretical understanding and practical performance of signal processing for nonlinear optical communications through machine learning

Deep Neural Network-Based Digital Pre-Distortion for High Baudrate Optical Coherent Transmission

Multi-Stage Machine Learning Enhanced DSP for DP-64QAM Coherent Optical Transmission Systems

Model-based Deep Learning for End-to-end Optimization in the Fiber-Terahertz Communication System

Joint Linear and Nonlinear Equalization Based on Cascaded ANN-MLSE with a Modified Loss Function for PAM-4 Optical Transmission

Coherent Optical Communications Enhanced by Machine Intelligence

An Interpretable Mapping from a Communication System to a Neural Network for Optimal Transceiver-Joint Equalization

End-to-end Deep Learning for a Flexible Coherent PON with User-Specific Constellation Optimization

Experimental Study of Deep Neural Network Equalizers Performance in Optical Links

Co-GRU Enhanced End-to-End Design for Long-haul Coherent Transmission Systems

Transformer-based Nonlinear Equalization for DP-16QAM Coherent Optical Communication Systems

Joint OSNR monitoring and modulation format identification in digital coherent receivers using deep neural networks

Convolutional Neural Network-Aided DP-64 QAM Coherent Optical Communication Systems

End-to-End Deep Learning for Long-haul Fiber Transmission Using Differentiable Surrogate Channel