Abstract:Precision and Recall are foundational metrics in machine learning where both accurate predictions and comprehensive coverage are essential, such as in recommender systems and multi-label learning. In these tasks, balancing precision (the proportion of relevant items among those predicted) and recall (the proportion of relevant items successfully predicted) is crucial. A key challenge is that one-sided feedback--where only positive examples are observed during training--is inherent in many practical problems. For instance, in recommender systems like YouTube, training data only consists of videos that a user has actively selected, while unselected items remain unseen. Despite this lack of negative feedback in training, avoiding undesirable recommendations at test time is essential. We introduce a PAC learning framework where each hypothesis is represented by a graph, with edges indicating positive interactions, such as between users and items. This framework subsumes the classical binary and multi-class PAC learning models as well as multi-label learning with partial feedback, where only a single random correct label per example is observed, rather than all correct labels. Our work uncovers a rich statistical and algorithmic landscape, with nuanced boundaries on what can and cannot be learned. Notably, classical methods like Empirical Risk Minimization fail in this setting, even for simple hypothesis classes with only two hypotheses. To address these challenges, we develop novel algorithms that learn exclusively from positive data, effectively minimizing both precision and recall losses. Specifically, in the realizable setting, we design algorithms that achieve optimal sample complexity guarantees. In the agnostic case, we show that it is impossible to achieve additive error guarantees--as is standard in PAC learning--and instead obtain meaningful multiplicative approximations.

Soft Precision and Recall

Evaluating Classification Systems Against Soft Labels with Fuzzy Precision and Recall

Sensitivity and Specificity versus Precision and Recall, and Related Dilemmas

Revisiting Precision and Recall Definition for Generative Model Evaluation

Improved Rule Based Rough Set Approach for Target Recognition

Precision and Recall Reject Curves for Classification

Enabling Precision/Recall Preferences for Semi-supervised SVM Training.

Two Kinds of Recall

Statistical Precision – Recall curves for object detection quality assessment

Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation

The Effect of Class Imbalance on Precision-Recall Curves

Precision at the indistinguishability threshold: a method for evaluating classification algorithms

The Representation of Meaningful Precision, and Accuracy

Probably Approximately Precision and Recall Learning

Classifier comparison using precision

Area under Precision-Recall Curves for Weighted and Unweighted Data

To be precise (imprecise) in utilitarian (hedonic) contexts: Examining the influence of numerical precision on consumer reactions to artificial intelligence‐based recommendations

Unifying and extending Precision Recall metrics for assessing generative models

Study on Roughness Measurement Model Based on Effect Precision

Precision and Semantics Based Fuzzy Identification