Abstract:Inference problems with conjectured statistical-computational gaps are ubiquitous throughout modern statistics, computer science and statistical physics. While there has been success evidencing these gaps from the failure of restricted classes of algorithms, progress towards a more traditional reduction-based approach to computational complexity in statistical inference has been limited. Existing reductions have largely been limited to inference problems with similar structure -- primarily mapping among problems representable as a sparse submatrix signal plus a noise matrix, which are similar to the common hardness assumption of planted clique. The insight in this work is that a slight generalization of the planted clique conjecture -- secret leakage planted clique -- gives rise to a variety of new average-case reduction techniques, yielding a web of reductions among problems with very different structure. Using variants of the planted clique conjecture for specific forms of secret leakage planted clique, we deduce tight statistical-computational tradeoffs for a diverse range of problems including robust sparse mean estimation, mixtures of sparse linear regressions, robust sparse linear regression, tensor PCA, variants of dense $k$-block stochastic block models, negatively correlated sparse PCA, semirandom planted dense subgraph, detection in hidden partition models and a universality principle for learning sparse mixtures. In particular, a $k$-partite hypergraph variant of the planted clique conjecture is sufficient to establish all of our computational lower bounds. Our techniques also reveal novel connections to combinatorial designs and to random matrix theory. This work gives the first evidence that an expanded set of hardness assumptions, such as for secret leakage planted clique, may be a key first step towards a more complete theory of reductions among statistical problems.

Machinery for Proving Sum-of-Squares Lower Bounds on Certification Problems

The power of sum-of-squares for detecting hidden structures

Sum-of-Squares Lower Bounds for the Minimum Circuit Size Problem

Does Subset Sum Admit Short Proofs?

Fourier sum of squares certificates

Sum-of-Squares Lower Bounds for Independent Set in Ultra-Sparse Random Graphs

Hypercontractivity, Sum-of-squares Proofs, and Their Applications

A Generalized Method for Proving Polynomial Calculus Degree Lower Bounds

Approximability and proof complexity

Certifying Euclidean Sections and Finding Planted Sparse Vectors Beyond the $\sqrt{n}$ Dimension Threshold

Feasibly Constructive Proof of Schwartz-Zippel Lemma and the Complexity of Finding Hitting Sets

Conic programming to understand sums of squares of eigenvalues of graphs

Nonlinear Random Matrices and Applications to the Sum of Squares Hierarchy

Arithmetic Circuit Lower Bounds via MaxRank

Sum-of-squares Chordal Decomposition of Polynomial Matrix Inequalities

SoS Certifiability of Subgaussian Distributions and its Algorithmic Applications

Positivity-preserving extensions of sum-of-squares pseudomoments over the hypercube

Computing sparse Fourier sum of squares on finite abelian groups in quasi-linear time

Certificates in P and Subquadratic-Time Computation of Radius, Diameter, and all Eccentricities in Graphs

Analysis of the Min-Sum Algorithm for Packing and Covering Problems via Linear Programming

Reducibility and Statistical-Computational Gaps from Secret Leakage