Abstract:Machine learning models are increasingly being utilized across various fields and tasks due to their outstanding performance and strong generalization capabilities. Nonetheless, their success hinges on the availability of large volumes of annotated data, the creation of which is often labor-intensive, time-consuming, and expensive. Many active learning (AL) approaches have been proposed to address these challenges, but they often fail to fully leverage the information from the core phases of AL, such as training on the labeled set and querying new unlabeled samples. To bridge this gap, we propose a novel AL approach, Loss Prediction Loss with Gradient Norm (LPLgrad), designed to quantify model uncertainty effectively and improve the accuracy of image classification tasks. LPLgrad operates in two distinct phases: (i) {\em Training Phase} aims to predict the loss for input features by jointly training a main model and an auxiliary model. Both models are trained on the labeled data to maximize the efficiency of the learning process, an aspect often overlooked in previous AL methods. This dual-model approach enhances the ability to extract complex input features and learn intrinsic patterns from the data effectively; (ii) {\em Querying Phase} that quantifies the uncertainty of the main model to guide sample selection. This is achieved by calculating the gradient norm of the entropy values for samples in the unlabeled dataset. Samples with the highest gradient norms are prioritized for labeling and subsequently added to the labeled set, improving the model's performance with minimal labeling effort. Extensive evaluations on real-world datasets demonstrate that the LPLgrad approach outperforms state-of-the-art methods by order of magnitude in terms of accuracy on a small number of labeled images, yet achieving comparable training and querying times in multiple image classification tasks.

Smooth Sailing: Improving Active Learning for Pre-trained Language Models with Representation Smoothness Analysis

FreeAL: Towards Human-Free Active Learning in the Era of Large Language Models

Beyond Active Learning: Leveraging the Full Potential of Human Interaction via Auto-Labeling, Human Correction, and Human Verification

Language Model-Driven Data Pruning Enables Efficient Active Learning

From Robustness to Improved Generalization and Calibration in Pre-trained Language Models

Practical Obstacles to Deploying Active Learning

Active Learning for Vision-Language Models

On the Fragility of Active Learners for Text Classification

Active Learning for NLP with Large Language Models

Investigating the Effectiveness of Representations Based on Pretrained Transformer-based Language Models in Active Learning for Labelling Text Datasets

On Dataset Transferability in Active Learning for Transformers

Avoid Wasted Annotation Costs in Open-set Active Learning with Pre-trained Vision-Language Model

Competence-Based Analysis of Language Models

Meta-Learning the Difference: Preparing Large Language Models for Efficient Adaptation

ALVIN: Active Learning Via INterpolation

APAM: Adaptive Pre-training and Adaptive Meta Learning in Language Model for Noisy Labels and Long-tailed Learning

Optimizing Active Learning for Low Annotation Budgets

LPLgrad: Optimizing Active Learning Through Gradient Norm Sample Selection and Auxiliary Model Training

Pedagogical Alignment of Large Language Models

Cartography Active Learning

On the Limitations of Simulating Active Learning