Abstract:Active learning (AL) aims to enable training high performance classifiers with low annotation cost by predicting which subset of unlabelled instances would be most beneficial to label. The importance of AL has motivated extensive research, proposing a wide variety of manually designed AL algorithms with diverse theoretical and intuitive motivations. In contrast to this body of research, we propose to treat active learning algorithm design as a meta-learning problem and learn the best criterion from data. We model an active learning algorithm as a deep neural network that inputs the base learner state and the unlabelled point set and predicts the best point to annotate next. Training this active query policy network with reinforcement learning, produces the best non-myopic policy for a given dataset. The key challenge in achieving a general solution to AL then becomes that of learner generalisation, particularly across heterogeneous datasets. We propose a multi-task dataset-embedding approach that allows dataset-agnostic active learners to be trained. Our evaluation shows that AL algorithms trained in this way can directly generalise across diverse problems.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the generality and adaptability in algorithm design in Active Learning (AL). Specifically, traditional active learning methods are usually manually designed. Although these methods each have their own theoretical and intuitive motivations, no single method can perform excellently on all datasets. Therefore, the paper proposes a new method, that is, to automatically learn the optimal active learning strategy through Deep Reinforcement Learning (DRL), rather than manually designing criteria for specific datasets or tasks. The goal of this method is to train an active learning query strategy that can generalize across different datasets and tasks. The main contributions of the paper are as follows: 1. **Regarding active learning algorithm design as a Meta - Learning problem**: Represent the active learning algorithm through a Deep Neural Network (DNN), input the current set of unlabeled data points and the state of the base learner, and output the next optimal labeled point. Use Reinforcement Learning (RL) to train this query strategy network to obtain the optimal non - myopic strategy for a given dataset. 2. **Solving the cross - dataset generalization problem**: In order to enable the learned strategy to generalize to unseen datasets, the paper proposes a multi - task dataset - embedding method, which allows training active learners without relying on specific datasets. This method adjusts the strategy by generating dataset embeddings so that it can adapt to the statistical characteristics of different datasets. 3. **Implementing an end - to - end query strategy**: Generate weight matrices through the Meta - Network, and these weight matrices parameterize the main strategy network, so that the entire query strategy can generalize between datasets in different - dimensional feature spaces. In summary, this paper aims to automatically learn active learning strategies through deep reinforcement learning, solve the problem of inconsistent performance of traditional methods on different datasets, and achieve broader applicability and better performance.

Meta-Learning Transferable Active Learning Policies by Deep Reinforcement Learning

A meta-framework for multi-label active learning based on deep reinforcement learning

Probabilistic Active Meta-Learning

Contextual Policy Transfer in Meta-Reinforcement Learning via Active Learning.

Discovering General-Purpose Active Learning Strategies

Reinforcement Teaching

AutoAL: Automated Active Learning with Differentiable Query Strategy Search

Image Classification with Deep Reinforcement Active Learning

A meta-active learning approach exploiting instance importance

A Survey of Deep Active Learning

Meta Reinforcement Learning with Task Embedding and Shared Policy.

Guided Meta-Policy Search

MAML2: meta reinforcement learning via meta-learning for task categories

Learning Generalizable Batch Active Learning Strategies Via Deep Q-networks (student Abstract).

Learning a Policy for Opportunistic Active Learning

Composite Active Learning: Towards Multi-Domain Active Learning with Theoretical Guarantees

Unsupervised Meta-Learning for Reinforcement Learning

ImitAL: Learned Active Learning Strategy on Synthetic Data

Meta-Reinforcement Learning with Dynamic Adaptiveness Distillation

Double Meta-Learning for Data Efficient Policy Optimization in Non-Stationary Environments

Meta Learning Shared Hierarchies