Abstract:Classical supervised machine learning techniques have been explored for semantically annotating unstructured textual data such as consumers' comments archived at social media websites to extract business intelligence. However, these techniques often require a large number of manually labeled training examples to produce accurate annotations. Several active learning approaches that are designed based on probabilistic sequence models have been explored to minimize the number of labeled training examples for semantic annotation tasks. Recent research has shown that large-margin classifiers are viable alternatives to automated semantic annotation, given their strong generalization capabilities and the ability to process high-dimensional data. However, the existing active learning methods that are designed for probabilistic sequence models cannot be easily adapted and applied to large-margin classifiers. The main contribution of this paper is the development of novel active learning methods for large-margin classifiers to fill the aforementioned research gap. In particular, we propose an innovative perspective of taking active learning as a search of optimal parameters for large-margin classifiers. A rigorous evaluation involving two benchmark tests and an empirical test based on real-world data extracted from Amazon.com reveals that the proposed active learning methods can train effective classifiers with significantly fewer training examples while achieving similar annotation performance, compared to a typical state-of-the-art classifier that only uses several labeled training examples. More specifically, one of our proposed active learning methods can reduce the number of training examples by 19.74% at the 68% level of F 1 when compared to the best baseline method, as evaluated based on the Amazon data set. Our research opens the door to the application of intelligent semantic annotation techniques to support real-world applications such as automatically analyzing consumer comments for customer relationship management.

Reducing Workload of Manual Annotation for User Requests Via a Novel Active Learning Framework

Uncertainty-aware Complementary Label Queries for Active Learning

Human-centred Design on Crowdsourcing Annotation Towards Improving Active Learning Model Performance

A Survey on Cost Types, Interaction Schemes, and Annotator Performance Models in Selection Algorithms for Active Learning in Classification

LEAF: A Less Expert Annotation Framework with Active Learning

FreeAL: Towards Human-Free Active Learning in the Era of Large Language Models

Effective Active Learning Strategies for the Use of Large-Margin Classifiers in Semantic Annotation: an Optimal Parameter Discovery Perspective.

Improving Active Learning by Data Balance to Reduce Annotation Efforts

Towards a SVM-struct Based Active Learning Algorithm for Least Cost Semantic Annotation

Annotation Cost Reduction of Stream-based Active Learning by Automated Weak Labeling using a Robot Arm

Literature survey of active learning in multimedia annotation and retrieval

Active Learning for NLP with Large Language Models

A Survey of Deep Active Learning

A Survey on Active Learning: State-of-the-Art, Practical Challenges and Research Directions

Beyond Active Learning: Leveraging the Full Potential of Human Interaction via Auto-Labeling, Human Correction, and Human Verification

Graph-based Active Semi-Supervised Learning: A New Perspective for Relieving Multi-Class Annotation Labor

Addressing practical challenges in Active Learning via a hybrid query strategy

Semi-automatic Video Annotation Based on Active Learning with Multiple Complementary Predictors

Enhancing Text Classification through LLM-Driven Active Learning and Human Annotation

A New Framework to Reduce Doctor’s Workload for Medical Image Annotation

An Empirical Study on the Efficacy of Deep Active Learning Techniques