Abstract:We present a framework for speeding up the time it takes to sample from discrete distributions $\mu$ defined over subsets of size $k$ of a ground set of $n$ elements, in the regime $k\ll n$. We show that having estimates of marginals $\mathbb{P}_{S\sim \mu}[i\in S]$, the task of sampling from $\mu$ can be reduced to sampling from distributions $\nu$ supported on size $k$ subsets of a ground set of only $n^{1-\alpha}\cdot \operatorname{poly}(k)$ elements. Here, $1/\alpha\in [1, k]$ is the parameter of entropic independence for $\mu$. Further, the sparsified distributions $\nu$ are obtained by applying a sparse (mostly $0$) external field to $\mu$, an operation that often retains algorithmic tractability of sampling from $\nu$. This phenomenon, which we dub domain sparsification, allows us to pay a one-time cost of estimating the marginals of $\mu$, and in return reduce the amortized cost needed to produce many samples from the distribution $\mu$, as is often needed in upstream tasks such as counting and inference. For a wide range of distributions where $\alpha=\Omega(1)$, our result reduces the domain size, and as a corollary, the cost-per-sample, by a $\operatorname{poly}(n)$ factor. Examples include monomers in a monomer-dimer system, non-symmetric determinantal point processes, and partition-constrained Strongly Rayleigh measures. Our work significantly extends the reach of prior work of Anari and Dereziński who obtained domain sparsification for distributions with a log-concave generating polynomial (corresponding to $\alpha=1$). As a corollary of our new analysis techniques, we also obtain a less stringent requirement on the accuracy of marginal estimates even for the case of log-concave polynomials; roughly speaking, we show that constant-factor approximation is enough for domain sparsification, improving over $O(1/k)$ relative error established in prior work.

Subsampling under distributional constraints

Subsampling scaling: a theory about inference from partly observed systems

Projection-Uniform Subsampling Methods for Big Data

Using hierarchical information-theoretic criteria to optimize subsampling of extensive datasets

A sub-sampling algorithm preventing outliers

Subsampling Suffices for Adaptive Data Analysis

Scalable subsampling: computation, aggregation and inference

Subsampling at Information Theoretically Optimal Rates

Decompounding Under General Mixing Distributions

Robust Model Fitting Using Higher Than Minimal Subset Sampling

Subsampling for Big Data Linear Models with Measurement Errors

Optimal subsampling algorithm for composite quantile regression with distributed data

Domain Sparsification of Discrete Distributions using Entropic Independence

Accounting for outliers in optimal subsampling methods

Optimal subsampling designs

Provably Efficient Posterior Sampling for Sparse Linear Regression via Measure Decomposition

Quantization‐Based Latin Hypercube Sampling for Dependent Inputs With an Application to Sensitivity Analysis of Environmental Models

Adaptive multilevel subset simulation with selective refinement

A model robust sub-sampling approach for Generalised Linear Models in Big data settings

Insufficient Gibbs sampling

Incorporating Subsampling into Bayesian Models for High-Dimensional Spatial Data