Abstract:When we can not assume a large amount of annotated data , active learning is a good strategy. It consists in learning a model on a small amount of annotated data (annotation budget) and in choosing the best set of points to annotate in order to improve the previous model and gain in generalization. In deep learning, active learning is usually implemented as an iterative process in which successive deep models are updated via fine tuning, but it still poses some issues. First, the initial batch of annotated images has to be sufficiently large to train a deep model. Such an assumption is strong, especially when the total annotation budget is reduced. We tackle this issue by using an approach inspired by transfer learning. A pre-trained model is used as a feature extractor and only shallow classifiers are learned during the active iterations. The second issue is the effectiveness of probability or feature estimates of early models for AL task. Samples are generally selected for annotation using acquisition functions based only on the last learned model. We introduce a novel acquisition function which exploits the iterative nature of AL process to select samples in a more robust fashion. Samples for which there is a maximum shift towards uncertainty between the last two learned models predictions are favored. A diversification step is added to select samples from different regions of the classification space and thus introduces a representativeness component in our approach. Evaluation is done against competitive methods with three balanced and imbalanced datasets and outperforms them.

Stopping Criterion for Active Learning Based on Error Stability

Stopping Criterion for Active Learning with Model Stability.

Stability-Based Stopping Criterion for Active Learning

Confidence-based stopping criteria for active learning for data annotation

A Method for Stopping Active Learning Based on Stabilizing Predictions and the Need for User-Adjustable Stopping

Analysis of Stopping Active Learning based on Stabilizing Predictions

Active Learning: Problem Settings and Recent Developments

Active learning using localized generalization error of candidate sample as criterion

Bounds on the Generalization Error in Active Learning

Neural Active Learning on Heteroskedastic Distributions

Late Stopping: Avoiding Confidently Learning from Mislabeled Examples

Scalable Active Learning by Approximated Error Reduction

Optimizing Active Learning for Low Annotation Budgets

Targeted Active Learning for Bayesian Decision-Making

The Use of Unlabeled Data versus Labeled Data for Stopping Active Learning for Text Classification

Deep Active Learning with Adaptive Acquisition

Absolute convergence and error thresholds in non-active adaptive sampling

Improve Cost Efficiency of Active Learning over Noisy Dataset

An error-based stopping criterion for spherical decomposition-based adaptive Kriging model and rare event estimation

Active learning for level set estimation under cost-dependent input uncertainty

Noisy Early Stopping for Noisy Labels