Abstract:The Area Under the ROC Curve (AUC) is a widely used performance measure for imbalanced classification arising from many application domains where high-dimensional sparse data is abundant. In such cases, each d dimensional sample has only k non-zero features with k ≪ d, and data arrives sequentially in a streaming form. Current online AUC optimization algorithms have high per-iteration cost O(d) and usually produce non-sparse solutions in general, and hence are not suitable for handling the data challenge mentioned above. In this paper, we aim to directly optimize the AUC score for high-dimensional sparse datasets under online learning setting and propose a new algorithm, FTRL-AUC. Our proposed algorithm can process data in an online fashion with a much cheaper per-iteration cost O(k), making it amenable for high-dimensional sparse streaming data analysis. Our new algorithmic design critically depends on a novel reformulation of the U-statistics AUC objective function as the empirical saddle point reformulation, and the innovative introduction of the “lazy update” rule so that the per-iteration complexity is dramatically reduced from O(d) to O(k). Furthermore, FTRL-AUC can inherently capture sparsity more effectively by applying a generalized Follow-The-Regularized-Leader (FTRL) framework. Experiments on real-world datasets demonstrate that FTRL-AUC significantly improves both run time and model sparsity while achieving competitive AUC scores compared with the state-of-the-art methods. Comparison with the online learning method for logistic loss demonstrates that FTRL-AUC achieves higher AUC scores especially when datasets are imbalanced. Experiments on real-world datasets demonstrate that FTRL-AUC significantly improves both run time and model sparsity while achieving competitive AUC scores compared with the state-of-the-art methods. Comparison with the online learning method for logistic loss demonstrates that FTRL-AUC achieves higher AUC scores especially when datasets are imbalanced.

Tau-FPL: Tolerance-Constrained Learning in Linear Time

A Deep 3D Residual CNN for False-Positive Reduction in Pulmonary Nodule Detection

Pbde: An Effective Method For Filtering False Positive Boxes In Object Detection

False Positive Rate Control for Positive Unlabeled Learning

Less is More: Pseudo-Label Filtering for Continual Test-Time Adaptation

Constrained Classification and Ranking via Quantiles

Taming False Positives in Out-of-Distribution Detection with Human Feedback

Variable Selection via Adaptive False Negative Control in Linear Regression

SLCA++: Unleash the Power of Sequential Fine-tuning for Continual Learning with Pre-training

Controlling False Positive/Negative Rates for Deep-Learning-Based Prostate Cancer Detection on Multiparametric MR images

A New Procedure for Controlling False Discovery Rate in Large-Scale t-tests

Adaptive label thresholding methods for online multi-label classification.

Conformal Prediction for Deep Classifier via Label Ranking

Ranking Regularization for Critical Rare Classes: Minimizing False Positives at a High True Positive Rate

Online AUC Optimization for Sparse High-Dimensional Datasets

Online Control of the False Coverage Rate and False Sign Rate

A ℓ2, 1 Norm Regularized Multi-Kernel Learning for False Positive Reduction in Lung Nodule CAD

Threshold-Consistent Margin Loss for Open-World Deep Metric Learning

FPR Estimation for Fraud Detection in the Presence of Class-Conditional Label Noise

Learning Near-Optimal Cost-Sensitive Decision Policy for Object Detection

Data-Driven Estimation of the False Positive Rate of the Bayes Binary Classifier via Soft Labels