Abstract:In this work, the problem of communicating decisions of a classifier over a noisy channel is considered. With machine learning based models being used in variety of time-sensitive applications, transmission of these decisions in a reliable and timely manner is of significant importance. To this end, we study the scenario where a probability vector (representing the decisions of a classifier) at the transmitter, needs to be transmitted over a noisy channel. Assuming that the distortion between the original probability vector and the reconstructed one at the receiver is measured via f-divergence, we study the trade-off between transmission latency and the distortion. We completely analyze this trade-off using uniform, lattice, and sparse lattice-based quantization techniques to encode the probability vector by first characterizing bit budgets for each technique given a requirement on the allowed source distortion. These bounds are then combined with results from finite-blocklength literature to provide a framework for analyzing the effects of both quantization distortion and distortion due to decoding error probability (i.e., channel effects) on the incurred transmission latency. Our results show that there is an interesting interplay between source distortion (i.e., distortion for the probability vector measured via f-divergence) and the subsequent channel encoding/decoding parameters; and indicate that a joint design of these parameters is crucial to navigate the latency-distortion tradeoff. We study the impact of changing different parameters (e.g. number of classes, SNR, source distortion) on the latency-distortion tradeoff and perform experiments on AWGN and fading channels. Our results indicate that sparse lattice-based quantization is the most effective at minimizing latency across various regimes and for sparse, high-dimensional probability vectors (i.e., high number of classes).

Recovering a Message from an Incomplete Set of Noisy Fragments

Communicating over the Torn-Paper Channel

Achieving the Dirty Paper Channel Capacity with Scalar Lattices and Probabilistic Shaping

Capacity Results for the Noisy Shuffling Channel

Capacity of Noisy Permutation Channels

Capacity of Frequency-based Channels: Encoding Information in Molecular Concentrations

On the Capacity of Memoryless Channels with Synchronization Errors

Capacity Approximations for Insertion Channels with Small Insertion Probabilities

On the Throughput of Channels That Wear Out

On noisy duplication channels with Markov sources

On the Capacity of the Dirty Paper Channel with Fast Fading and Discrete Channel States

Capacity Definitions for General Channels with Receiver Side Information

Improved Upper and Lower Bounds on the Capacity of the Binary Deletion Channel

A characterization of the capacity of online (causal) binary channels

Latency-Distortion Tradeoffs in Communicating Classification Results over Noisy Channels

Capacity Results for Non-Ergodic Multi-Modal Broadcast Channels with Controllable Statistics

On the Outage Capacity of a Dying Channel

Shannon Capacity of Channels with Markov Insertions, Deletions and Substitutions

Finite Blocklength Performance of Capacity-achieving Codes in the Light of Complexity Theory

Discrete denoising for channels with memory

Binomial Channel: On the Capacity-Achieving Distribution and Bounds on the Capacity