Abstract:This paper addresses a fundamental problem in random variate generation: given access to a random source that emits a stream of independent fair bits, what is the most accurate and entropy-efficient algorithm for sampling from a discrete probability distribution $(p_1, \dots, p_n)$, where the probabilities of the output distribution $(\hat{p}_1, \dots, \hat{p}_n)$ of the sampling algorithm must be specified using at most $k$ bits of precision? We present a theoretical framework for formulating this problem and provide new techniques for finding sampling algorithms that are optimal both statistically (in the sense of sampling accuracy) and information-theoretically (in the sense of entropy consumption). We leverage these results to build a system that, for a broad family of measures of statistical accuracy, delivers a sampling algorithm whose expected entropy usage is minimal among those that induce the same distribution (i.e., is "entropy-optimal") and whose output distribution $(\hat{p}_1, \dots, \hat{p}_n)$ is a closest approximation to the target distribution $(p_1, \dots, p_n)$ among all entropy-optimal sampling algorithms that operate within the specified $k$-bit precision. This optimal approximate sampler is also a closer approximation than any (possibly entropy-suboptimal) sampler that consumes a bounded amount of entropy with the specified precision, a class which includes floating-point implementations of inversion sampling and related methods found in many software libraries. We evaluate the accuracy, entropy consumption, precision requirements, and wall-clock runtime of our optimal approximate sampling algorithms on a broad set of distributions, demonstrating the ways that they are superior to existing approximate samplers and establishing that they often consume significantly fewer resources than are needed by exact samplers.

Sampling Based Approximate \tau Τ -Quantile Computation Algorithm in Sensor Networks.

Distributed fault estimation of networked systems using quantized measurements

Sampling based algorithms for quantile computation in sensor networks.

Approximate Quantile Computation over Sensor Networks

Optimal Estimation Algorithm Design under Event-based Sensor Data Scheduling*

Bernoulli Sampling Based (element Of, Delta)-Approximate Aggregation in Large-Scale Sensor Networks.

Distributed Quantile Regression over Sensor Networks

Quantized Distributed Estimation with Adaptive Combiner

Bernoulli Sampling Based (Epsilon, Delta)-Approximate Frequency Query In Mobile Ad Hoc Networks

Tail quantile estimation for non-preemptive priority queues

Optimal Approximate Sampling from Discrete Probability Distributions

Enabling Epsilon-Approximate Querying in Sensor Networks

Distributed state and fault estimation over sensor networks with probabilistic quantizations: The dynamic event-triggered case

Energy-Efficient Probabilistic Area Coverage in Wireless Sensor Networks.

An Approximation Theory Framework for Measure-Transport Sampling Algorithms

A probabilistic approach to statistical QoS provision of event detection in sensor networks

A Fast Algorithm for Approximate Quantiles in High Speed Data Streams

Small Area Quantile Estimation

Distributed Adaptive Quantization for Wireless Sensor Networks: from Delta Modulation to Maximum Likelihood

When Quantum Computing Meets Database: A Hybrid Sampling Framework for Approximate Query Processing

Two maximum entropy based algorithms for running quantile estimation in non-stationary data streams