Abstract:Communication-constrained algorithms for decentralized learning and optimization rely on local updates coupled with the exchange of compressed signals. In this context, differential quantization is an effective technique to mitigate the negative impact of compression by leveraging correlations between successive iterates. In addition, the use of error feedback, which consists of incorporating the compression error into subsequent steps, is a powerful mechanism to compensate for the bias caused by the compression. Under error feedback, performance guarantees in the literature have so far focused on algorithms employing a fusion center or a special class of contractive compressors that cannot be implemented with a finite number of bits. In this work, we propose a new decentralized communication-efficient learning approach that blends differential quantization with error feedback. The approach is specifically tailored for decentralized learning problems where agents have individual risk functions to minimize subject to subspace constraints that require the minimizers across the network to lie in low-dimensional subspaces. This constrained formulation includes consensus or single-task optimization as special cases, and allows for more general task relatedness models such as multitask smoothness and coupled optimization. We show that, under some general conditions on the compression noise, and for sufficiently small step-sizes $\mu$, the resulting communication-efficient strategy is stable both in terms of mean-square error and average bit rate: by reducing $\mu$, it is possible to keep the estimation errors small (on the order of $\mu$) without increasing indefinitely the bit rate as $\mu\rightarrow 0$. The results establish that, in the small step-size regime and with a finite number of bits, it is possible to attain the performance achievable in the absence of compression.

Learning Physical-Layer Communication with Quantized Feedback

On the design of hybrid limited feedback for massive MIMO systems

Goodput Maximization with Quantized Feedback in the Finite Blocklength Regime for Quasi-Static Channels

Optimized Training and Feedback for MIMO Downlink Channels

Quantized iterative learning control of communication-constrained systems with encoding and decoding mechanism

Changeable Rate and Novel Quantization for CSI Feedback Based on Deep Learning

Distributed Jointly Sparse Bayesian Learning with Quantized Communication.

Data-Driven Control of Linear Systems via Quantized Feedback

A Tutorial on Quantized Feedback Control

Quantization Bits Allocation for Wireless Federated Learning

Differential error feedback for communication-efficient decentralized learning

Quantization Adaptor for Bit-Level Deep Learning-Based Massive MIMO CSI Feedback

Distributed Adaptation of Quantized Feedback for Downlink Network MIMO Systems

On the Impact of Quantized Channel Feedback in Guaranteeing Secrecy with Artificial Noise.

Design of Deep Learning-Based 1-bit Transceiver With Oversampling and Faster-Than-Nyquist (FTN) Signaling

Design and Analysis of Uplink and Downlink Communications for Federated Learning

Quantized Beamforming with Channel Prediction - Transactions Letters

Deep Task-Based Quantization

On zero-error communication via quantum channels in the presence of noiseless feedback

Bandlimited Communication with One-Bit Quantization and Oversampling: Transceiver Design and Performance Evaluation.

Data-driven Control of Unknown Linear Systems via Quantized Feedback