Abstract:Precision and Recall are foundational metrics in machine learning where both accurate predictions and comprehensive coverage are essential, such as in recommender systems and multi-label learning. In these tasks, balancing precision (the proportion of relevant items among those predicted) and recall (the proportion of relevant items successfully predicted) is crucial. A key challenge is that one-sided feedback--where only positive examples are observed during training--is inherent in many practical problems. For instance, in recommender systems like YouTube, training data only consists of videos that a user has actively selected, while unselected items remain unseen. Despite this lack of negative feedback in training, avoiding undesirable recommendations at test time is essential. We introduce a PAC learning framework where each hypothesis is represented by a graph, with edges indicating positive interactions, such as between users and items. This framework subsumes the classical binary and multi-class PAC learning models as well as multi-label learning with partial feedback, where only a single random correct label per example is observed, rather than all correct labels. Our work uncovers a rich statistical and algorithmic landscape, with nuanced boundaries on what can and cannot be learned. Notably, classical methods like Empirical Risk Minimization fail in this setting, even for simple hypothesis classes with only two hypotheses. To address these challenges, we develop novel algorithms that learn exclusively from positive data, effectively minimizing both precision and recall losses. Specifically, in the realizable setting, we design algorithms that achieve optimal sample complexity guarantees. In the agnostic case, we show that it is impossible to achieve additive error guarantees--as is standard in PAC learning--and instead obtain meaningful multiplicative approximations.

CS 6781-Theoretical Foundations of Machine Learning Lecture 5 : PAC Sample Complexity Proof

Measurability in the Fundamental Theorem of Statistical Learning

Proper vs Improper Quantum PAC learning

A Parameterized Theory of PAC Learning

On Several Problems of Statistical Learning Theory

Provable Advantage in Quantum PAC Learning

Probably Approximately Precision and Recall Learning

Error Exponent in Agnostic PAC Learning

Is Efficient PAC Learning Possible with an Oracle That Responds 'Yes' or 'No'?

Gödel Incompleteness Theorem for PAC Learnable Theory

Private PAC Learning May be Harder than Online Learning

Fat Shattering, Joint Measurability, and PAC Learnability of POVM Hypothesis Classes

Characterizing the Sample Complexity of Private Learners

Do PAC-Learners Learn the Marginal Distribution?

Structure discovery in PAC-learning by random projections

On the complexity of PAC learning in Hilbert spaces

A Labelled Sample Compression Scheme of Size at Most Quadratic in the VC Dimension

Distributional PAC-Learning from Nisan's Natural Proofs

Local Borsuk-Ulam, Stability, and Replicability

The Sample Complexity of Multi-Distribution Learning for VC Classes