Abstract:Training a classifier exploiting a huge amount of supervised data is expensive or even prohibited in a situation, where the labeling cost is high. The remarkable progress in working with weaker forms of supervision is binary classification from multiple unlabeled datasets which requires the knowledge of exact class priors for all unlabeled datasets. However, the availability of class priors is restrictive in many real-world scenarios. To address this issue, we propose to solve a new problem setting, i.e., binary classification from multiple unlabeled datasets with only one pairwise numerical relationship of class priors (MU-OPPO), which knows the relative order (which unlabeled dataset has a higher proportion of positive examples) of two class-prior probabilities for two datasets among multiple unlabeled datasets. In MU-OPPO, we do not need the class priors for all unlabeled datasets, but we only require that there exists a pair of unlabeled datasets for which we know which unlabeled dataset has a larger class prior. Clearly, this form of supervision is easier to be obtained, which can make labeling costs almost free. We propose a novel framework to handle the MU-OPPO problem, which consists of four sequential modules: (i) pseudo label assignment; (ii) confident example collection; (iii) class prior estimation; (iv) classifier training with estimated class priors. Theoretically, we analyze the gap between estimated class priors and true class priors under the proposed framework. Empirically, we confirm the superiority of our framework with comprehensive experiments. Experimental results demonstrate that our framework brings smaller estimation errors of class priors and better performance of binary classification.

AUC Optimization from Multiple Unlabeled Datasets.

Weakly Supervised AUC Optimization: A Unified Partial AUC Approach

Semi-Supervised AUC Optimization Without Guessing Labels of Unlabeled Data.

Learning with Multiclass AUC: Theory and Algorithms

AUL is a Better Optimization Metric in PU Learning

Robust AUC optimization under the supervision of clean data

AUC-Oriented Domain Adaptation: from Theory to Algorithm.

Evolutionary Multitasking AUC Optimization [Research Frontier]

When All We Need is a Piece of the Pie: A Generic Framework for Optimizing Two-way Partial AUC.

Dual Coordinate Descent Method for Solving AUC Optimization Problem

One-Pass AUC Optimization

Two-pass AUC optimization

A Boosting Method for Direct AUC Optimization

Stochastic Methods for AUC Optimization subject to AUC-based Fairness Constraints

Optimizing Two-Way Partial AUC with an End-to-End Framework

Doubly Robust AUC Optimization Against Noisy and Adversarial Samples

Does it pay to optimize AUC?

AUC Maximization in the Era of Big Data and AI: A Survey

Making Binary Classification from Multiple Unlabeled Datasets Almost Free of Supervision

Stochastic Optimization of Areas Under Precision-Recall Curves with Provable Convergence

Learning Personalized Attribute Preference Via Multi-Task AUC Optimization.