Abstract:State-of-the-art high-spectral-efficiency communication systems employ high-order modulation formats coupled with high symbol rates to accommodate the ever-growing demand for data rate-hungry applications. However, such systems are more vulnerable to linear and nonlinear transmission impairments, and it is important to mitigate the performance loss via digital signal processing. In this paper, we propose a novel machine learning approach for blind channel equalization and estimation using the vector quantized (VQ) \ac{VAE} framework. The proposed approach generalizes the applicability of the conventional \ac{VAE}-based equalizer to nonlinear systems employing high-order modulation formats by introducing a codebook component and an associated novel loss function. We evaluate the performance of the proposed method over a linear additive white Gaussian noise channel with intersymbol interference and two nonlinear scenarios. Simulation results show that the proposed method can achieve similar performance as a data aided equalizer using the \acf{MMSE} criterion, and outperforms the blind\ac{CMA} and the \ac{VAE}-based channel equalizer. Furthermore, we show that for the linear channel, the proposed scheme exhibits better convergence properties than the \ac{MMSE}-based, the \ac{CMA}-based, and the \ac{VAE}-based equalizers in terms of both convergence speed and robustness to variations in training batch size and learning rate.

An ICA and EC Based Approach for Blind Equalization and Channel Parameter Estimation

Blind Equalization and Channel Estimation in Coherent Optical Communications Using Variational Autoencoders

Fast Blind Channel Equalization Based on Online Deep Neural Network

Linear Precoding Aided Blind Equalization with Independent Component Analysis in MIMO OFDM Systems

A New Dual Mode Blind Equalization Algorithm

Blind Channel Equalization using Variational Autoencoders

Blind Channel Equalization Using Vector-Quantized Variational Autoencoders

Bi-iterative least squares algorithms for blind channel identification and equalization with second-order statistics

Constrained Optimization Methods for Blind Equalization of Multiple Fir Channels

Adaptive Minimum Variance Methods for Direct Blind Multichannel Equalization

A Trellis-Based Technique for Blind Channel Estimation and Equalization.

A Variable Step Size Blind Equalization Algorithm for QAM Signals

DL-NA-SBD: an Unsupervised Online Deep Learning Approach for Blind Channel Equalization

A New Blind Equalization Algorithm Based On Convex Combination

Robust Blind Channel Equalization Based on Input Decision Information.

Unsupervised Linear and Nonlinear Channel Equalization and Decoding using Variational Autoencoders

On Blind Equalization of Simo Time-Varying Channels

Blind Deconvolution/Equalization Using State-Space Models

Blind and Channel-agnostic Equalization Using Adversarial Networks

Validation of a Green Wireless Communication System with ICA Based Semi-Blind Equalization

Modified Baum-Welch Algorithm for Joint Blind Channel Estimation and Turbo Equalization