Compressibility and probabilistic proofs

Alexander Shen

DOI: https://doi.org/10.48550/arXiv.1703.03342

2017-03-10

Abstract:We consider several examples of probabilistic existence proofs using compressibility arguments, including some results that involve Lovász local lemma.

Discrete Mathematics,Combinatorics

What problem does this paper attempt to address?

Formal Probabilistic Methods for Combinatorial Structures using the Lovász Local Lemma

Chelsea Edmonds,Lawrence C. Paulson

DOI: https://doi.org/10.1145/3636501.3636946

2024-01-08

Abstract:Formalised libraries of combinatorial mathematics have rapidly expanded over the last five years, but few use one of the most important tools: probability. How can often intuitive probabilistic arguments on the existence of combinatorial structures, such as hypergraphs, be translated into a formal text? We present a modular framework using locales in Isabelle/HOL to formalise such probabilistic proofs, including the basic existence method and first formalisation of the Lovász local lemma, a fundamental result in probability. The formalisation focuses on general, reusable formal probabilistic lemmas for combinatorial structures, and highlights several notable gaps in typical intuitive probabilistic reasoning on paper. The applicability of the techniques is demonstrated through the formalisation of several classic lemmas on the existence of hypergraphs with certain colourings.

Logic in Computer Science,Combinatorics
Existence proofs in combinatorics using independence

D. Ilyinskiy,A. Raigorodskiy,A. Skopenkov

DOI: https://doi.org/10.48550/arXiv.1411.3171

2014-11-12

History and Overview

Abstract:This note is purely expository and is in Russian. We show how to prove interesting combinatorial results using the local Lovasz lemma. The note is accessible for students having basic knowledge of combinatorics; the notion of independence is defined and the Lovasz lemma is stated and proved. Our exposition follows `Probabilistic methods' of N. Alon and J. Spencer. The main difference is that we show how the proof could have been invented. The material is presented as a sequence of problems, which is peculiar not only to Zen monasteries but also to advanced mathematical education; most problems are presented with hints or solutions.
Entropic Compressibility of Lévy Processes

Julien Fageot,Alireza Fallah,Thibaut Horel

DOI: https://doi.org/10.1109/TIT.2022.3167863

2022-05-16

Abstract:In contrast to their seemingly simple and shared structure of independence and stationarity, Lévy processes exhibit a wide variety of behaviors, from the self-similar Wiener process to piecewise-constant compound Poisson processes. Inspired by the recent paper of Ghourchian, Amini, and Gohari (2018), we characterize their compressibility by studying the entropy of their double discretization (both in time and amplitude) in the regime of vanishing discretization steps. For a Lévy process with absolutely continuous marginals, this reduces to understanding the asymptotics of the differential entropy of its marginals at small times, for which we obtain a new local central limit theorem. We generalize known results for stable processes to the non-stable case, with a special focus on Lévy processes that are locally self-similar, and conceptualize a new compressibility hierarchy of Lévy processes, captured by their Blumenthal-Getoor index.

Probability,Information Theory
An Algorithmic Proof of the Lovasz Local Lemma via Resampling Oracles

Nicholas J.A. Harvey,Jan Vondrak

DOI: https://doi.org/10.1109/focs.2015.85

2015-10-01

Abstract:The Lovász Local Lemma is a seminal result in probabilistic combinatorics. It gives a sufficient condition on a probability space and a collection of events for the existence of an outcome that simultaneously avoids all of those events. Finding such an outcome by an efficient algorithm has been an active research topic for decades. Breakthrough work of Moser and Tardos (2009) presented an efficient algorithm for a general setting primarily characterized by a product structure on the probability space. In this work we present an efficient algorithm for a much more general setting. Our main assumption is that there exist certain functions, called resampling oracles, that can be invoked to address the undesired occurrence of the events. We show that, in all scenarios to which the original Lovász Local Lemma applies, there exist resampling oracles, although they are not necessarily efficient. Nevertheless, for essentially all known applications of the Lovász Local Lemma and its generalizations, we have designed efficient resampling oracles. As applications of these techniques, we present new results for packings of Latin transversals, rainbow matchings and rainbow spanning trees.
Probabilistic Couplings for Probabilistic Reasoning

Justin Hsu

DOI: https://doi.org/10.48550/arXiv.1710.09951

2017-11-01

Abstract:This thesis explores proofs by coupling from the perspective of formal verification. Long employed in probability theory and theoretical computer science, these proofs construct couplings between the output distributions of two probabilistic processes. Couplings can imply various guarantees comparing two runs of a probabilistic computation. We first show that proofs in the program logic pRHL describe couplings. We formalize couplings that establish various probabilistic properties, including distribution equivalence, convergence, and stochastic domination. Then we give a proofs-as-programs interpretation: a coupling proof encodes a probabilistic product program, whose properties imply relational properties of the original programs. We design the logic xpRHL to construct the product, with extensions to model shift coupling and path coupling. We then propose an approximate version of probabilistic coupling and a corresponding proof technique---proof by approximate coupling---inspired by the logic apRHL, a version of pRHL for building approximate liftings. Drawing on ideas from existing privacy proofs, we extend apRHL with novel proof rules for constructing new approximate couplings. We give an approximate coupling proof of privacy for the Sparse Vector mechanism, a well-known algorithm from the privacy literature whose privacy proof is notoriously subtle, and produce the first formalized proof of privacy for Sparse Vector in apRHL. Finally, we propose several more sophisticated constructions for approximate couplings: a principle for showing accuracy-dependent privacy, a generalization of the advanced composition theorem, and an optimal approximate coupling relating two subsets. We also show equivalences between our approximate couplings and other existing definitions. These ingredients support the first formalized proof of privacy for the Between Thresholds mechanism.

Logic in Computer Science,Data Structures and Algorithms,Programming Languages
A probabilistic proof of a lemma that is not Burnside's

Vince Vatter

DOI: https://doi.org/10.1080/00029890.2020.1676619

2020-07-30

Abstract:We give a probabilistic proof of the orbit-counting lemma.

History and Overview
An Explicit Croot-Łaba-Sisask Lemma Free of Probabilistic Language

Olivier Ramaré

DOI: https://doi.org/10.1007/s00574-024-00397-5

2024-04-27

Bulletin of the Brazilian Mathematical Society New Series

Abstract:We provide an explicit and probabilistic language-free proof of the famous Croot-Łaba-Sisask Lemma. In between, we do the same for the Khintchine and Marcinkiewicz-Zygmund inequalities and explicitate the implied constants for the upper bounds.

mathematics
Kolmogorov complexity as a combinatorial tool

Alexander Shen

2024-05-15

Abstract:Kolmogorov complexity is often used as a convenient language for counting and/or probabilistic existence proofs. However, there are some applications where Kolmogorov complexity is used in a more subtle way. We provide one (somehow) surprising example where an existence of a winning strategy in a natural combinatorial game is proven (and no direct proof is known).

Discrete Mathematics,Information Theory,Combinatorics
The Round Complexity of Proofs in the Bounded Quantum Storage Model

Alex B. Grilo,Philippe Lamontagne

2024-05-28

Abstract:The round complexity of interactive proof systems is a key question of practical and theoretical relevance in complexity theory and cryptography. Moreover, results such as QIP = QIP(3) (STOC'00) show that quantum resources significantly help in such a task. In this work, we initiate the study of round compression of protocols in the bounded quantum storage model (BQSM). In this model, the malicious parties have a bounded quantum memory and they cannot store the all the qubits that are transmitted in the protocol. Our main results in this setting are the following: 1. There is a non-interactive (statistical) witness indistinguishable proof for any language in NP (and even QMA) in BQSM in the plain model. We notice that in this protocol, only the memory of the verifier is bounded. 2. Any classical proof system can be compressed in a two-message quantum proof system in BQSM. Moreover, if the original proof system is zero-knowledge, the quantum protocol is zero-knowledge too. In this result, we assume that the prover has bounded memory. Finally, we give evidence towards the "tightness" of our results. First, we show that NIZK in the plain model against BQS adversaries is unlikely with standard techniques. Second, we prove that without the BQS model there is no 2-message zero-knowledge quantum interactive proof, even under computational assumptions.

Quantum Physics,Computational Complexity,Cryptography and Security
Tight constraints on probabilistic convertibility of quantum states

Bartosz Regula

DOI: https://doi.org/10.22331/q-2022-09-22-817

2023-06-22

Abstract:We develop two general approaches to characterising the manipulation of quantum states by means of probabilistic protocols constrained by the limitations of some quantum resource theory. First, we give a general necessary condition for the existence of a physical transformation between quantum states, obtained using a recently introduced resource monotone based on the Hilbert projective metric. In all affine quantum resource theories (e.g. coherence, asymmetry, imaginarity) as well as in entanglement distillation, we show that the monotone provides a necessary and sufficient condition for one-shot resource convertibility under resource-non-generating operations, and hence no better restrictions on all probabilistic protocols are possible. We use the monotone to establish improved bounds on the performance of both one-shot and many-copy probabilistic resource distillation protocols. Complementing this approach, we introduce a general method for bounding achievable probabilities in resource transformations under resource-non-generating maps through a family of convex optimisation problems. We show it to tightly characterise single-shot probabilistic distillation in broad types of resource theories, allowing an exact analysis of the trade-offs between the probabilities and errors in distilling maximally resourceful states. We demonstrate the usefulness of both of our approaches in the study of quantum entanglement distillation.

Quantum Physics,Mathematical Physics
Quantum Proof Systems for Iterated Exponential Time, and Beyond.

Joseph Fitzsimons,Zhengfeng Ji,Thomas Vidick,Henry Yuen

DOI: https://doi.org/10.1145/3313276.3316343

2018-01-01

Abstract:We show that any language solvable in nondeterministic time exp( exp(⋯exp( n ))), where the number of iterated exponentials is an arbitrary function R ( n ), can be decided by a multiprover interactive proof system with a classical polynomial-time verifier and a constant number of quantum entangled provers, with completeness 1 and soundness 1 − exp(− C exp(⋯exp( n ))), where the number of iterated exponentials is R ( n )−1 and C >0 is a universal constant. The result was previously known for R =1 and R =2; we obtain it for any time-constructible function R . The result is based on a compression technique for interactive proof systems with entangled provers that significantly simplifies and strengthens a protocol compression result of Ji (STOC’17). As a separate consequence of this technique we obtain a different proof of Slofstra’s recent result on the uncomputability of the entangled value of multiprover games (Forum of Mathematics, Pi 2019). Finally, we show that even minor improvements to our compression result would yield remarkable consequences in computational complexity theory and the foundations of quantum mechanics: first, it would imply that the class MIP* contains all computable languages; second, it would provide a negative resolution to a multipartite version of Tsirelson’s problem on the relation between the commuting operator and tensor product models for quantum correlations.
On statistical learning via the lens of compression

Ofir David,Shay Moran,Amir Yehudayoff

DOI: https://doi.org/10.48550/arXiv.1610.03592

2016-12-31

Abstract:This work continues the study of the relationship between sample compression schemes and statistical learning, which has been mostly investigated within the framework of binary classification. The central theme of this work is establishing equivalences between learnability and compressibility, and utilizing these equivalences in the study of statistical learning theory. We begin with the setting of multiclass categorization (zero/one loss). We prove that in this case learnability is equivalent to compression of logarithmic sample size, and that uniform convergence implies compression of constant size. We then consider Vapnik's general learning setting: we show that in order to extend the compressibility-learnability equivalence to this case, it is necessary to consider an approximate variant of compression. Finally, we provide some applications of the compressibility-learnability equivalences: (i) Agnostic-case learnability and realizable-case learnability are equivalent in multiclass categorization problems (in terms of sample complexity). (ii) This equivalence between agnostic-case learnability and realizable-case learnability does not hold for general learning problems: There exists a learning problem whose loss function takes just three values, under which agnostic-case and realizable-case learnability are not equivalent. (iii) Uniform convergence implies compression of constant size in multiclass categorization problems. Part of the argument includes an analysis of the uniform convergence rate in terms of the graph dimension, in which we improve upon previous bounds. (iv) A dichotomy for sample compression in multiclass categorization problems: If a non-trivial compression exists then a compression of logarithmic size exists. (v) A compactness theorem for multiclass categorization problems.

Machine Learning,Discrete Mathematics,Logic in Computer Science,Combinatorics,Logic
Non-Commutative Formulas and Frege Lower Bounds: a New Characterization of Propositional Proofs.

Fu Li,Iddo Tzameret,Zhengyu Wang

DOI: https://doi.org/10.4230/lipics.ccc.2015.412

2014-01-01

Abstract:Does every Boolean tautology have a short propositional-calculus proof? Here, a propositional-calculus (i.e. Frege) proof is any proof starting from a set of axioms and deriving new Boolean formulas using a fixed set of sound derivation rules. Establishing any super-polynomial size lower bound on Frege proofs (in terms of the size of the formula proved) is a major open problem in proof complexity, and among a handful of fundamental hardness questions in complexity theory by and large. Non-commutative arithmetic formulas, on the other hand, constitute a quite weak computational model, for which exponential-size lower bounds were shown already back in 1991 by Nisan [20], using a particularly transparent argument. In this work we show that Frege lower bounds in fact follow from corresponding size lower bounds on non-commutative formulas computing certain polynomials (and that such lower bounds on non-commutative formulas must exist, unless NP=coNP ). More precisely, we demonstrate a natural association between tautologies T to non-commutative polynomials p , such that: •if T has a polynomial-size Frege proof then p has a polynomial-size non-commutative arithmetic formula; and conversely, when T is a DNF, if p has a polynomial-size non-commutative arithmetic formula over GF (2) then T has a Frege proof of quasi-polynomial size. The argument is a characterization of Frege proofs as non-commutative formulas: we show that the Frege system is (quasi-) polynomially equivalent to a non-commutative Ideal Proof System (IPS), following the recent work of Grochow and Pitassi [10] that introduced a propositional proof system in which proofs are arithmetic circuits, and the work in [35] that considered adding the commutator as an axiom in algebraic propositional proof systems. This gives a characterization of propositional Frege proofs in terms of (non-commutative) arithmetic formulas that is tighter than (the formula version of IPS) in Grochow and Pitassi [10], in the following sense: (i) The non-commutative IPS is polynomial-time checkable -- whereas the original IPS was checkable in probabilistic polynomial-time; and (ii) Frege proofs unconditionally quasi-polynomially simulate the non-commutative IPS -- whereas Frege was shown to efficiently simulate IPS only assuming that the decidability of PIT for (commutative) arithmetic formulas by polynomial-size circuits is efficiently provable in Frege.
Generalized Rényi entropy accumulation theorem and generalized quantum probability estimation

Amir Arqand,Thomas A. Hahn,Ernest Y.-Z. Tan

2024-10-27

Abstract:The entropy accumulation theorem, and its subsequent generalized version, is a powerful tool in the security analysis of many device-dependent and device-independent cryptography protocols. However, it has the drawback that the finite-size bounds it yields are not necessarily optimal, and furthermore it relies on the construction of an affine min-tradeoff function, which can often be challenging to construct optimally in practice. In this work, we address both of these challenges simultaneously by deriving a new entropy accumulation bound. Our bound yields significantly better finite-size performance, and can be computed as an intuitively interpretable convex optimization, without any specification of affine min-tradeoff functions. Furthermore, it can be applied directly at the level of Rényi entropies if desired, yielding fully-Rényi security proofs. Our proof techniques are based on elaborating on a connection between entropy accumulation and the frameworks of quantum probability estimation or $f$-weighted Rényi entropies, and in the process we obtain some new results with respect to those frameworks as well. In particular, those findings imply that our bounds apply to prepare-and-measure protocols without the virtual tomography procedures or repetition-rate restrictions previously required for entropy accumulation.

Quantum Physics
Towards an optimal hypergraph container lemma

Marcelo Campos,Wojciech Samotij

2024-09-19

Abstract:The hypergraph container lemma is a powerful tool in probabilistic combinatorics that has found many applications since it was first proved a decade ago. Roughly speaking, it asserts that the family of independent sets of every uniform hypergraph can be covered by a small number of almost-independent sets, called containers. In this article, we formulate and prove two new versions of the lemma that display the following three attractive features. First, they both admit short and simple proofs that have surprising connections to other well-studied topics in probabilistic combinatorics. Second, they use alternative notions of almost-independence in order to describe the containers. Third, they yield improved dependence of the number of containers on the uniformity of the hypergraph, hitting a natural barrier for second-moment-type approaches.

Combinatorics
Lower Bounds for Possibly Divergent Probabilistic Programs.

Shenghua Feng,Mingshuai Chen,Han Su,Benjamin Lucien Kaminski,Joost-Pieter Katoen,Naijun Zhan

DOI: https://doi.org/10.1145/3586051

2023-01-01

Proceedings of the ACM on Programming Languages

Abstract:We present a new proof rule for verifying lower bounds on quantities of probabilistic programs. Our proof rule is not confined to almost-surely terminating programs - as is the case for existing rules - and can be used to establish non-trivial lower bounds on, e.g., termination probabilities and expected values, for possibly divergent probabilistic loops, e.g., the well-known three-dimensional random walk on a lattice.
How to Verify Any (Reasonable) Distribution Property: Computationally Sound Argument Systems for Distributions

Tal Herman,Guy Rothblum

2024-09-10

Abstract:As statistical analyses become more central to science, industry and society, there is a growing need to ensure correctness of their results. Approximate correctness can be verified by replicating the entire analysis, but can we verify without replication? Building on a recent line of work, we study proof-systems that allow a probabilistic verifier to ascertain that the results of an analysis are approximately correct, while drawing fewer samples and using less computational resources than would be needed to replicate the analysis. We focus on distribution testing problems: verifying that an unknown distribution is close to having a claimed property. Our main contribution is a interactive protocol between a verifier and an untrusted prover, which can be used to verify any distribution property that can be decided in polynomial time given a full and explicit description of the distribution. If the distribution is at statistical distance $\varepsilon$ from having the property, then the verifier rejects with high probability. This soundness property holds against any polynomial-time strategy that a cheating prover might follow, assuming the existence of collision-resistant hash functions (a standard assumption in cryptography). For distributions over a domain of size $N$, the protocol consists of $4$ messages and the communication complexity and verifier runtime are roughly $\widetilde{O}\left(\sqrt{N} / \varepsilon^2 \right)$. The verifier's sample complexity is $\widetilde{O}\left(\sqrt{N} / \varepsilon^2 \right)$, and this is optimal up to $\polylog(N)$ factors (for any protocol, regardless of its communication complexity). Even for simple properties, approximately deciding whether an unknown distribution has the property can require quasi-linear sample complexity and running time. For any such property, our protocol provides a quadratic speedup over replicating the analysis.

Computational Complexity,Cryptography and Security
Coarse embeddability, $L^1$-compression and Percolations on General Graphs

Chiranjib Mukherjee,Konstantin Recke

2024-07-30

Abstract:We show that a locally finite, connected graph has a coarse embedding into a Hilbert space if and only if there exist bond percolations with arbitrarily large marginals and two-point function vanishing at infinity. We further show that the decay of the two-point function is stretched exponential with stretching exponent $\alpha\in[0,1]$ if and only if the $L^1$-compression exponent of the graph is at least $\alpha$, leading to a probabilistic characterization of this exponent. These results are new even in the particular setting of Cayley graphs of finitely generated groups. The proofs build on new probabilistic methods introduced recently by the authors to study group-invariant percolation on Cayley graphs [25,26], which are now extended to the general, non-symmetric situation of graphs to study their coarse embeddability and $L^1$-compression exponents.

Probability,Metric Geometry
Constraint-Based Synthesis of Coupling Proofs

Aws Albarghouthi,Justin Hsu

DOI: https://doi.org/10.48550/arXiv.1804.04052

2018-04-11

Programming Languages

Abstract:Proof by coupling is a classical technique for proving properties about pairs of randomized algorithms by carefully relating (or coupling) two probabilistic executions. In this paper, we show how to automatically construct such proofs for probabilistic programs. First, we present f-coupled postconditions, an abstraction describing two correlated program executions. Second, we show how properties of f-coupled postconditions can imply various probabilistic properties of the original programs. Third, we demonstrate how to reduce the proof-search problem to a purely logical synthesis problem of the form $\exists f\ldotp \forall X\ldotp \phi$, making probabilistic reasoning unnecessary. We develop a prototype implementation to automatically build coupling proofs for probabilistic properties, including uniformity and independence of program expressions.

Compressibility and probabilistic proofs

Formal Probabilistic Methods for Combinatorial Structures using the Lovász Local Lemma

Existence proofs in combinatorics using independence

Entropic Compressibility of Lévy Processes

An Algorithmic Proof of the Lovasz Local Lemma via Resampling Oracles

Probabilistic Couplings for Probabilistic Reasoning

A probabilistic proof of a lemma that is not Burnside's

An Explicit Croot-Łaba-Sisask Lemma Free of Probabilistic Language

Kolmogorov complexity as a combinatorial tool

The Round Complexity of Proofs in the Bounded Quantum Storage Model

Tight constraints on probabilistic convertibility of quantum states

Quantum Proof Systems for Iterated Exponential Time, and Beyond.

On statistical learning via the lens of compression

Non-Commutative Formulas and Frege Lower Bounds: a New Characterization of Propositional Proofs.

Generalized Rényi entropy accumulation theorem and generalized quantum probability estimation

Towards an optimal hypergraph container lemma

Lower Bounds for Possibly Divergent Probabilistic Programs.

How to Verify Any (Reasonable) Distribution Property: Computationally Sound Argument Systems for Distributions

Coarse embeddability, $L^1$-compression and Percolations on General Graphs

Constraint-Based Synthesis of Coupling Proofs