Abstract:We give the first agnostic, efficient, proper learning algorithm for monotone Boolean functions. Given $2^{\tilde{O}(\sqrt{n}/\varepsilon)}$ uniformly random examples of an unknown function $f:\{\pm 1\}^n \rightarrow \{\pm 1\}$, our algorithm outputs a hypothesis $g:\{\pm 1\}^n \rightarrow \{\pm 1\}$ that is monotone and $(\mathrm{opt} + \varepsilon)$-close to $f$, where $\mathrm{opt}$ is the distance from $f$ to the closest monotone function. The running time of the algorithm (and consequently the size and evaluation time of the hypothesis) is also $2^{\tilde{O}(\sqrt{n}/\varepsilon)}$, nearly matching the lower bound of Blais et al (RANDOM '15). We also give an algorithm for estimating up to additive error $\varepsilon$ the distance of an unknown function $f$ to monotone using a run-time of $2^{\tilde{O}(\sqrt{n}/\varepsilon)}$. Previously, for both of these problems, sample-efficient algorithms were known, but these algorithms were not run-time efficient. Our work thus closes this gap in our knowledge between the run-time and sample complexity. This work builds upon the improper learning algorithm of Bshouty and Tamon (JACM '96) and the proper semiagnostic learning algorithm of Lange, Rubinfeld, and Vasilyan (FOCS '22), which obtains a non-monotone Boolean-valued hypothesis, then ``corrects'' it to monotone using query-efficient local computation algorithms on graphs. This black-box correction approach can achieve no error better than $2\mathrm{opt} + \varepsilon$ information-theoretically; we bypass this barrier by a) augmenting the improper learner with a convex optimization step, and b) learning and correcting a real-valued function before rounding its values to Boolean. Our real-valued correction algorithm solves the ``poset sorting'' problem of [LRV22] for functions over general posets with non-Boolean labels.

Agnostically Learning Boolean Functions with Finite Polynomial Representation.

Agnostic proper learning of monotone functions: beyond the black-box correction barrier

Agnostic PAC Learning of k-juntas Using L2-Polynomial Regression

Learning Concepts Definable in First-Order Logic with Counting

Testably Learning Polynomial Threshold Functions

Learning Polynomial Problems with $SL(2,\mathbb{R})$ Equivariance

From Exact Learning to Computing Boolean Functions and Back Again

Agnostic $Q$-learning with Function Approximation in Deterministic Systems - Near-Optimal Bounds on Approximation Error and Sample Complexity.

Agnostic Learning of Arbitrary ReLU Activation under Gaussian Marginals

On Learning with LAD

Learning Algorithms via Neural Logic Networks

Learning Boolean functions with concentrated spectra

The Polynomial Method is Universal for Distribution-Free Correlational SQ Learning

Approximating the Number of Relevant Variables in a Parity Implies Proper Learning

NIL: Learning Nonlinear Interpolants

Boolean Logic as an Error feedback mechanism

Error Exponent in Agnostic PAC Learning

Learning Implicitly with Noisy Data in Linear Arithmetic

Hardness of Learning Boolean Functions from Label Proportions

Agnostic Learnability of Halfspaces via Logistic Loss

Agnostic Membership Query Learning with Nontrivial Savings: New Results, Techniques