Quantization of Recurrent Neural Network for Low-Complexity High-Speed IM/DD System Equalization Based on Neuron Clustering

Zhaopeng Xu,Honglin Ji,Yu Yang,Gang Qiao,Qi Wu,Weiqi Lu,Lulu Liu,Shangcheng Wang,Junpeng Liang,Jiali Li,Jinlong Wei,Zhixue He,Weisheng Hu,William Shieh
DOI: https://doi.org/10.1109/acp/poem59049.2023.10369904
2023-01-01
Abstract:Neural networks (NNs) are widely employed as effective equalizers in intensity-modulated direct-detection (IM/DD) links due to their excellent ability in dealing with nonlinear channel impairments. However, the complexity concern impedes the real-time application of NN-based receivers. To address this issue, we propose mixed-precision quantization of recurrent NN (RNN)-based equalizers in a 100-Gb/s 15-km C-band IM/DD system, which saves about 73.3% and 22.4% memory compared with traditional floating-point-based and fixed-precision quantized RNN. A simple and effective neuron clustering approach is proposed to realize mixed-precision quantization of RNN without degrading system performance.
What problem does this paper attempt to address?