Abstract:Carmosino et al. (2016) demonstrated that natural proofs of circuit lower bounds for $\Lambda$ imply efficient algorithms for learning $\Lambda$-circuits, but only over \textit{the uniform distribution}, with \textit{membership queries}, and provided $\AC^0[p] \subseteq \Lambda$. We consider whether this implication can be generalized to $\Lambda \not\supseteq \AC^0[p]$, and to learning algorithms which use only random examples and learn over arbitrary example distributions (Valiant's PAC-learning model). We first observe that, if, for any circuit class $\Lambda$, there is an implication from natural proofs for $\Lambda$ to PAC-learning for $\Lambda$, then standard assumptions from lattice-based cryptography do not hold. In particular, we observe that depth-2 majority circuits are a (conditional) counter example to the implication, since Nisan (1993) gave a natural proof, but Klivans and Sherstov (2009) showed hardness of PAC-learning under lattice-based assumptions. We thus ask: what learning algorithms can we reasonably expect to follow from Nisan's natural proofs? Our main result is that all natural proofs arising from a type of communication complexity argument, including Nisan's, imply PAC-learning algorithms in a new \textit{distributional} variant (i.e., an ``average-case'' relaxation) of Valiant's PAC model. Our distributional PAC model is stronger than the average-case prediction model of Blum et al. (1993) and the heuristic PAC model of Nanashima (2021), and has several important properties which make it of independent interest, such as being \textit{boosting-friendly}. The main applications of our result are new distributional PAC-learning algorithms for depth-2 majority circuits, polytopes and DNFs over natural target distributions, as well as the nonexistence of encoded-input weak PRFs that can be evaluated by depth-2 majority circuits.

New Cryptographic Hardness for Learning Intersections of Halfspaces over Boolean Cubes with Membership Queries

Improved Hardness Results for Learning Intersections of Halfspaces

Agnostic PAC Learning of k-juntas Using L2-Polynomial Regression

On PAC Learning Halfspaces in Non-interactive Local Privacy Model with Public Unlabeled Data

Fat Shattering, Joint Measurability, and PAC Learnability of POVM Hypothesis Classes

Provable Advantage in Quantum PAC Learning

Hardness of Learning Boolean Functions from Label Proportions

Learning low-degree quantum objects

Distributional PAC-Learning from Nisan's Natural Proofs

Private PAC Learning May be Harder than Online Learning

Efficient PAC Learning of Halfspaces with Constant Malicious Noise Rate

Monotone probability distributions over the Boolean cube can be learned with sublinear samples

Testably Learning Polynomial Threshold Functions

On the non-efficient PAC learnability of conjunctive queries

On the complexity of PAC learning in Hilbert spaces

Fooling intersections of low-weight halfspaces

The Polynomial Method is Universal for Distribution-Free Correlational SQ Learning

Near-Optimal Active Learning of Halfspaces via Query Synthesis in the Noisy Setting

Reliable Learning of Halfspaces under Gaussian Marginals

Local Borsuk-Ulam, Stability, and Replicability