Bayesian Active Learning Using Arbitrary Binary Valued Queries

Liu Yang,Steve Hanneke,Jaime Carbonell

DOI: https://doi.org/10.1007/978-3-642-16108-7_8

2010-01-01

Abstract:We explore a general Bayesian active learning setting, in which the learner can ask arbitrary yes/no questions. We derive upper and lower bounds on the expected number of queries required to achieve a specified expected risk.

What problem does this paper attempt to address?

Uncertainty-aware Complementary Label Queries for Active Learning

Shengyuan Liu,Ke Chen,Tianlei Hu,Yunqing Mao

DOI: https://doi.org/10.1631/fitee.2200589

2023-01-01

Abstract:Many active learning methods are based on the assumption that a learner simply asks for the true labels of some training data from annotators. Unfortunately, it is expensive to exactly annotate instances in real-world classification tasks. To mitigate this problem, we propose a novel active learning setting, named active learning with complementary labels (ALCL). The ALCL learners only ask yes/no questions in some classes. After receiving answers from annotators, ALCL learners get a few supervised data and more training instances with complementary labels, which only specify one of the classes to which the pattern does not belong. There are two challenging issues in ALCL: the first is how to learn from these complementary labels, and another is how to sample instances to be queried. For the first issue, we propose a novel method that redistributes the weights of instances based on the balance of category contribution to learn from ordinary labels and complementary labels. For the second issue, we propose a weighting mechanism to improve existing uncertainty-based sampling strategies under this novel setup. Experiments on various datasets demonstrate the feasibility of ALCL and the superiority of our approaches.
Bounds on the Generalization Error in Active Learning

Vincent Menden,Yahya Saleh,Armin Iske

2024-09-10

Abstract:We establish empirical risk minimization principles for active learning by deriving a family of upper bounds on the generalization error. Aligning with empirical observations, the bounds suggest that superior query algorithms can be obtained by combining both informativeness and representativeness query strategies, where the latter is assessed using integral probability metrics. To facilitate the use of these bounds in application, we systematically link diverse active learning scenarios, characterized by their loss functions and hypothesis classes to their corresponding upper bounds. Our results show that regularization techniques used to constraint the complexity of various hypothesis classes are sufficient conditions to ensure the validity of the bounds. The present work enables principled construction and empirical quality-evaluation of query algorithms in active learning.

Machine Learning
Active Learning of Bayesian Linear Models with High-Dimensional Binary Features by Parameter Confidence-Region Estimation

Yu Inatsu,Masayuki Karasuyama,Keiichi Inoue,Hideki Kandori,Ichiro Takeuchi

DOI: https://doi.org/10.1162/neco_a_01310

IF: 3.278

2020-10-01

Neural Computation

Abstract:In this letter, we study an active learning problem for maximizing an unknown linear function with high-dimensional binary features. This problem is notoriously complex but arises in many important contexts. When the sampling budget, that is, the number of possible function evaluations, is smaller than the number of dimensions, it tends to be impossible to identify all of the optimal binary features. Therefore, in practice, only a small number of such features are considered, with the majority kept fixed at certain default values, which we call the working set heuristic. The main contribution of this letter is to formally study the working set heuristic and present a suite of theoretically robust algorithms for more efficient use of the sampling budget. Technically, we introduce a novel method for estimating the confidence regions of model parameters that is tailored to active learning with high-dimensional binary features. We provide a rigorous theoretical analysis of these algorithms and prove that a commonly used working set heuristic can identify optimal binary features with favorable sample complexity. We explore the performance of the proposed approach through numerical simulations and an application to a functional protein design problem.

computer science, artificial intelligence,neurosciences
Deep Bayesian Active Learning, A Brief Survey on Recent Advances

Salman Mohamadi,Hamidreza Amindavar

DOI: https://doi.org/10.48550/arXiv.2012.08044

2022-04-21

Abstract:Active learning frameworks offer efficient data annotation without remarkable accuracy degradation. In other words, active learning starts training the model with a small size of labeled data while exploring the space of unlabeled data in order to select most informative samples to be labeled. Generally speaking, representing the uncertainty is crucial in any active learning framework, however, deep learning methods are not capable of either representing or manipulating model uncertainty. On the other hand, from the real world application perspective, uncertainty representation is getting more and more attention in the machine learning community. Deep Bayesian active learning frameworks and generally any Bayesian active learning settings, provide practical consideration in the model which allows training with small data while representing the model uncertainty for further efficient training. In this paper, we briefly survey recent advances in Bayesian active learning and in particular deep Bayesian active learning frameworks.

Machine Learning
Query Efficient Posterior Estimation in Scientific Experiments via Bayesian Active Learning

Kirthevasan Kandasamy,Jeff Schneider,Barnabás Póczos

DOI: https://doi.org/10.1016/j.artint.2016.11.002

2017-02-04

Abstract:A common problem in disciplines of applied Statistics research such as Astrostatistics is of estimating the posterior distribution of relevant parameters. Typically, the likelihoods for such models are computed via expensive experiments such as cosmological simulations of the universe. An urgent challenge in these research domains is to develop methods that can estimate the posterior with few likelihood evaluations. In this paper, we study active posterior estimation in a Bayesian setting when the likelihood is expensive to evaluate. Existing techniques for posterior estimation are based on generating samples representative of the posterior. Such methods do not consider efficiency in terms of likelihood evaluations. In order to be query efficient we treat posterior estimation in an active regression framework. We propose two myopic query strategies to choose where to evaluate the likelihood and implement them using Gaussian processes. Via experiments on a series of synthetic and real examples we demonstrate that our approach is significantly more query efficient than existing techniques and other heuristics for posterior estimation.

Machine Learning
Dynamic Exploration-Exploitation Trade-Off in Active Learning Regression with Bayesian Hierarchical Modeling

Upala Junaida Islam,Kamran Paynabar,George Runger,Ashif Sikandar Iquebal

DOI: https://doi.org/10.48550/arXiv.2304.07665

2023-10-01

Abstract:Active learning provides a framework to adaptively query the most informative experiments towards learning an unknown black-box function. Various approaches of active learning have been proposed in the literature, however, they either focus on exploration or exploitation in the design space. Methods that do consider exploration-exploitation simultaneously employ fixed or ad-hoc measures to control the trade-off that may not be optimal. In this paper, we develop a Bayesian hierarchical approach, referred as BHEEM, to dynamically balance the exploration-exploitation trade-off as more data points are queried. To sample from the posterior distribution of the trade-off parameter, We subsequently formulate an approximate Bayesian computation approach based on the linear dependence of queried data in the feature space. Simulated and real-world examples show the proposed approach achieves at least 21% and 11% average improvement when compared to pure exploration and exploitation strategies respectively. More importantly, we note that by optimally balancing the trade-off between exploration and exploitation, BHEEM performs better or at least as well as either pure exploration or pure exploitation.

Machine Learning
Near-Optimal Bayesian Active Learning with Noisy Observations

Daniel Golovin,Andreas Krause,Debajyoti Ray

DOI: https://doi.org/10.48550/arXiv.1010.3091

2013-12-16

Abstract:We tackle the fundamental problem of Bayesian active learning with noise, where we need to adaptively select from a number of expensive tests in order to identify an unknown hypothesis sampled from a known prior distribution. In the case of noise-free observations, a greedy algorithm called generalized binary search (GBS) is known to perform near-optimally. We show that if the observations are noisy, perhaps surprisingly, GBS can perform very poorly. We develop EC2, a novel, greedy active learning algorithm and prove that it is competitive with the optimal policy, thus obtaining the first competitiveness guarantees for Bayesian active learning with noisy observations. Our bounds rely on a recently discovered diminishing returns property called adaptive submodularity, generalizing the classical notion of submodular set functions to adaptive policies. Our results hold even if the tests have non-uniform cost and their noise is correlated. We also propose EffECXtive, a particularly fast approximation of EC2, and evaluate it on a Bayesian experimental design problem involving human subjects, intended to tease apart competing economic theories of how people make decisions under uncertainty.

Machine Learning,Artificial Intelligence,Data Structures and Algorithms
Active Learning with Simple Questions

Vasilis Kontonis,Mingchen Ma,Christos Tzamos

2024-06-10

Abstract:We consider an active learning setting where a learner is presented with a pool S of n unlabeled examples belonging to a domain X and asks queries to find the underlying labeling that agrees with a target concept h^* \in H. In contrast to traditional active learning that queries a single example for its label, we study more general region queries that allow the learner to pick a subset of the domain T \subset X and a target label y and ask a labeler whether h^*(x) = y for every example in the set T \cap S. Such more powerful queries allow us to bypass the limitations of traditional active learning and use significantly fewer rounds of interactions to learn but can potentially lead to a significantly more complex query language. Our main contribution is quantifying the trade-off between the number of queries and the complexity of the query language used by the learner. We measure the complexity of the region queries via the VC dimension of the family of regions. We show that given any hypothesis class H with VC dimension d, one can design a region query family Q with VC dimension O(d) such that for every set of n examples S \subset X and every h^* \in H, a learner can submit O(d log n) queries from Q to a labeler and perfectly label S. We show a matching lower bound by designing a hypothesis class H with VC dimension d and a dataset S \subset X of size n such that any learning algorithm using any query class with VC dimension less than O(d) must make poly(n) queries to label S perfectly. Finally, we focus on well-studied hypothesis classes including unions of intervals, high-dimensional boxes, and d-dimensional halfspaces, and obtain stronger results. In particular, we design learning algorithms that (i) are computationally efficient and (ii) work even when the queries are not answered based on the learner's pool of examples S but on some unknown superset L of S

Machine Learning,Data Structures and Algorithms
Deep Bayesian Active Learning for Natural Language Processing: Results of a Large-Scale Empirical Study

Aditya Siddhant,Zachary C. Lipton

DOI: https://doi.org/10.48550/arXiv.1808.05697

2018-08-16

Computation and Language

Abstract:Several recent papers investigate Active Learning (AL) for mitigating the data dependence of deep learning for natural language processing. However, the applicability of AL to real-world problems remains an open question. While in supervised learning, practitioners can try many different methods, evaluating each against a validation set before selecting a model, AL affords no such luxury. Over the course of one AL run, an agent annotates its dataset exhausting its labeling budget. Thus, given a new task, an active learner has no opportunity to compare models and acquisition functions. This paper provides a large scale empirical study of deep active learning, addressing multiple tasks and, for each, multiple datasets, multiple models, and a full suite of acquisition functions. We find that across all settings, Bayesian active learning by disagreement, using uncertainty estimates provided either by Dropout or Bayes-by Backprop significantly improves over i.i.d. baselines and usually outperforms classic uncertainty sampling.
Online active classification via margin-based and feature-based label queries

Tingting Zhai,Frédéric Koriche,Yang Gao,Junwu Zhu,Bin Li

DOI: https://doi.org/10.1007/s10994-022-06133-8

IF: 5.414

2022-03-11

Machine Learning

Abstract:In the paradigm of online active classification, the learner not only has to predict the label of each incoming instance, but also must decide whether the true label of that instance should be supplied, or not. The overall goal is to minimize the number of prediction mistakes with few label queries. In this paper, we focus on a novel framework for online active learning, with the aim of handling high dimensional classification problems. The key component of our framework is to exploit both the margin-based predictive uncertainty and the feature-based discriminative information of the current instance, in order to determine whether it should be labeled. Based on this labeling strategy, we propose several online active learning algorithms, for both binary classification tasks and multiclass ones. For these algorithms, which use adaptive subgradient methods for updating their linear model, expected mistake bounds are provided. Experiments on high-dimensional (binary and multiclass) classification datasets reveal the benefit of our label query strategy, and show the superiority of our algorithms over the existing ones.

computer science, artificial intelligence
Active classification with comparison queries

Daniel M. Kane,Shachar Lovett,Shay Moran,Jiapeng Zhang

DOI: https://doi.org/10.48550/arXiv.1704.03564

2017-06-02

Abstract:We study an extension of active learning in which the learning algorithm may ask the annotator to compare the distances of two examples from the boundary of their label-class. For example, in a recommendation system application (say for restaurants), the annotator may be asked whether she liked or disliked a specific restaurant (a label query); or which one of two restaurants did she like more (a comparison query). We focus on the class of half spaces, and show that under natural assumptions, such as large margin or bounded bit-description of the input examples, it is possible to reveal all the labels of a sample of size $n$ using approximately $O(\log n)$ queries. This implies an exponential improvement over classical active learning, where only label queries are allowed. We complement these results by showing that if any of these assumptions is removed then, in the worst case, $\Omega(n)$ queries are required. Our results follow from a new general framework of active learning with additional queries. We identify a combinatorial dimension, called the \emph{inference dimension}, that captures the query complexity when each additional query is determined by $O(1)$ examples (such as comparison queries, each of which is determined by the two compared examples). Our results for half spaces follow by bounding the inference dimension in the cases discussed above.

Machine Learning,Computational Geometry
Bayesian active learning for choice models with deep Gaussian processes

Jie Yang,Diego Klabjan

DOI: https://doi.org/10.48550/arXiv.1805.01867

2018-05-05

Abstract:In this paper, we propose an active learning algorithm and models which can gradually learn individual's preference through pairwise comparisons. The active learning scheme aims at finding individual's most preferred choice with minimized number of pairwise comparisons. The pairwise comparisons are encoded into probabilistic models based on assumptions of choice models and deep Gaussian processes. The next-to-compare decision is determined by a novel acquisition function. We benchmark the proposed algorithm and models using functions with multiple local optima and one public airline itinerary dataset. The experiments indicate the effectiveness of our active learning algorithm and models.

Machine Learning
Deep Bayesian Active Learning for Preference Modeling in Large Language Models

Luckeciano C. Melo,Panagiotis Tigas,Alessandro Abate,Yarin Gal

2024-10-28

Abstract:Leveraging human preferences for steering the behavior of Large Language Models (LLMs) has demonstrated notable success in recent years. Nonetheless, data selection and labeling are still a bottleneck for these systems, particularly at large scale. Hence, selecting the most informative points for acquiring human feedback may considerably reduce the cost of preference labeling and unleash the further development of LLMs. Bayesian Active Learning provides a principled framework for addressing this challenge and has demonstrated remarkable success in diverse settings. However, previous attempts to employ it for Preference Modeling did not meet such expectations. In this work, we identify that naive epistemic uncertainty estimation leads to the acquisition of redundant samples. We address this by proposing the Bayesian Active Learner for Preference Modeling (BAL-PM), a novel stochastic acquisition policy that not only targets points of high epistemic uncertainty according to the preference model but also seeks to maximize the entropy of the acquired prompt distribution in the feature space spanned by the employed LLM. Notably, our experiments demonstrate that BAL-PM requires 33% to 68% fewer preference labels in two popular human preference datasets and exceeds previous stochastic Bayesian acquisition policies.

Machine Learning,Computation and Language
Dirichlet Active Learning

Kevin Miller,Ryan Murray

2023-11-10

Abstract:This work introduces Dirichlet Active Learning (DiAL), a Bayesian-inspired approach to the design of active learning algorithms. Our framework models feature-conditional class probabilities as a Dirichlet random field and lends observational strength between similar features in order to calibrate the random field. This random field can then be utilized in learning tasks: in particular, we can use current estimates of mean and variance to conduct classification and active learning in the context where labeled data is scarce. We demonstrate the applicability of this model to low-label rate graph learning by constructing ``propagation operators'' based upon the graph Laplacian, and offer computational studies demonstrating the method's competitiveness with the state of the art. Finally, we provide rigorous guarantees regarding the ability of this approach to ensure both exploration and exploitation, expressed respectively in terms of cluster exploration and increased attention to decision boundaries.

Machine Learning
Bayesian Active Learning for Discrete Latent Variable Models

Aditi Jha,Zoe C. Ashwood,Jonathan W. Pillow

DOI: https://doi.org/10.1162/neco_a_01646

2023-06-03

Abstract:Active learning seeks to reduce the amount of data required to fit the parameters of a model, thus forming an important class of techniques in modern machine learning. However, past work on active learning has largely overlooked latent variable models, which play a vital role in neuroscience, psychology, and a variety of other engineering and scientific disciplines. Here we address this gap by proposing a novel framework for maximum-mutual-information input selection for discrete latent variable regression models. We first apply our method to a class of models known as "mixtures of linear regressions" (MLR). While it is well known that active learning confers no advantage for linear-Gaussian regression models, we use Fisher information to show analytically that active learning can nevertheless achieve large gains for mixtures of such models, and we validate this improvement using both simulations and real-world data. We then consider a powerful class of temporally structured latent variable models given by a Hidden Markov Model (HMM) with generalized linear model (GLM) observations, which has recently been used to identify discrete states from animal decision-making data. We show that our method substantially reduces the amount of data needed to fit GLM-HMM, and outperforms a variety of approximate methods based on variational and amortized inference. Infomax learning for latent variable models thus offers a powerful for characterizing temporally structured latent states, with a wide variety of applications in neuroscience and beyond.

Machine Learning,Neurons and Cognition
Improving Active Learning with a Bayesian Representation of Epistemic Uncertainty

Jake Thomas,Jeremie Houssineau

2024-12-11

Abstract:A popular strategy for active learning is to specifically target a reduction in epistemic uncertainty, since aleatoric uncertainty is often considered as being intrinsic to the system of interest and therefore not reducible. Yet, distinguishing these two types of uncertainty remains challenging and there is no single strategy that consistently outperforms the others. We propose to use a particular combination of probability and possibility theories, with the aim of using the latter to specifically represent epistemic uncertainty, and we show how this combination leads to new active learning strategies that have desirable properties. In order to demonstrate the efficiency of these strategies in non-trivial settings, we introduce the notion of a possibilistic Gaussian process (GP) and consider GP-based multiclass and binary classification problems, for which the proposed methods display a strong performance for both simulated and real datasets.

Methodology,Machine Learning
Learning Thresholds with Latent Values and Censored Feedback

Jiahao Zhang,Tao Lin,Weiqiang Zheng,Zhe Feng,Yifeng Teng,Xiaotie Deng

DOI: https://doi.org/10.48550/arxiv.2312.04653

2023-01-01

Abstract:In this paper, we investigate a problem of *actively* learning threshold in latent space, where the *unknown* reward $g(\gamma, v)$ depends on the proposed threshold $\gamma$ and latent value $v$ and it can be $only$ achieved if the threshold is lower than or equal to the *unknown* latent value. This problem has broad applications in practical scenarios, e.g., reserve price optimization in online auctions, online task assignments in crowdsourcing, setting recruiting bars in hiring, etc. We first characterize the query complexity of learning a threshold with the expected reward at most $\epsilon$ smaller than the optimum and prove that the number of queries needed can be infinitely large even when $g(\gamma, v)$ is monotone with respect to both $\gamma$ and $v$. On the positive side, we provide a tight query complexity $\tilde{\Theta}(1/\epsilon^3)$ when $g$ is monotone and the CDF of value distribution is Lipschitz. Moreover, we show a tight $\tilde{\Theta}(1/\epsilon^3)$ query complexity can be achieved as long as $g$ satisfies one-sided Lipschitzness, which provides a complete characterization for this problem. Finally, we extend this model to an online learning setting and demonstrate a tight $\Theta(T^{2/3})$ regret bound using continuous-arm bandit techniques and the aforementioned query complexity results.
Near-Optimal Active Learning of Halfspaces via Query Synthesis in the Noisy Setting

Lin Chen,Hamed Hassani,Amin Karbasi

DOI: https://doi.org/10.48550/arXiv.1603.03515

2016-11-13

Abstract:In this paper, we consider the problem of actively learning a linear classifier through query synthesis where the learner can construct artificial queries in order to estimate the true decision boundaries. This problem has recently gained a lot of interest in automated science and adversarial reverse engineering for which only heuristic algorithms are known. In such applications, queries can be constructed de novo to elicit information (e.g., automated science) or to evade detection with minimal cost (e.g., adversarial reverse engineering). We develop a general framework, called dimension coupling (DC), that 1) reduces a d-dimensional learning problem to d-1 low dimensional sub-problems, 2) solves each sub-problem efficiently, 3) appropriately aggregates the results and outputs a linear classifier, and 4) provides a theoretical guarantee for all possible schemes of aggregation. The proposed method is proved resilient to noise. We show that the DC framework avoids the curse of dimensionality: its computational complexity scales linearly with the dimension. Moreover, we show that the query complexity of DC is near optimal (within a constant factor of the optimum algorithm). To further support our theoretical analysis, we compare the performance of DC with the existing work. We observe that DC consistently outperforms the prior arts in terms of query complexity while often running orders of magnitude faster.

Artificial Intelligence,Information Theory,Machine Learning
Bayesian Binary Search

Vikash Singh,Matthew Khanzadeh,Vincent Davis,Harrison Rush,Emanuele Rossi,Jesse Shrader,Pietro Lio

2024-10-03

Abstract:We present Bayesian Binary Search (BBS), a novel probabilistic variant of the classical binary search/bisection algorithm. BBS leverages machine learning/statistical techniques to estimate the probability density of the search space and modifies the bisection step to split based on probability density rather than the traditional midpoint, allowing for the learned distribution of the search space to guide the search algorithm. Search space density estimation can flexibly be performed using supervised probabilistic machine learning techniques (e.g., Gaussian process regression, Bayesian neural networks, quantile regression) or unsupervised learning algorithms (e.g., Gaussian mixture models, kernel density estimation (KDE), maximum likelihood estimation (MLE)). We demonstrate significant efficiency gains of using BBS on both simulated data across a variety of distributions and in a real-world binary search use case of probing channel balances in the Bitcoin Lightning Network, for which we have deployed the BBS algorithm in a production setting.

Machine Learning
Bounded Memory Active Learning through Enriched Queries

Max Hopkins,Daniel Kane,Shachar Lovett,Michal Moshkovitz

DOI: https://doi.org/10.48550/arXiv.2102.05047

2021-02-10

Abstract:The explosive growth of easily-accessible unlabeled data has lead to growing interest in active learning, a paradigm in which data-hungry learning algorithms adaptively select informative examples in order to lower prohibitively expensive labeling costs. Unfortunately, in standard worst-case models of learning, the active setting often provides no improvement over non-adaptive algorithms. To combat this, a series of recent works have considered a model in which the learner may ask enriched queries beyond labels. While such models have seen success in drastically lowering label costs, they tend to come at the expense of requiring large amounts of memory. In this work, we study what families of classifiers can be learned in bounded memory. To this end, we introduce a novel streaming-variant of enriched-query active learning along with a natural combinatorial parameter called lossless sample compression that is sufficient for learning not only with bounded memory, but in a query-optimal and computationally efficient manner as well. Finally, we give three fundamental examples of classifier families with small, easy to compute lossless compression schemes when given access to basic enriched queries: axis-aligned rectangles, decision trees, and halfspaces in two dimensions.

Machine Learning

Bayesian Active Learning Using Arbitrary Binary Valued Queries

Uncertainty-aware Complementary Label Queries for Active Learning

Bounds on the Generalization Error in Active Learning

Active Learning of Bayesian Linear Models with High-Dimensional Binary Features by Parameter Confidence-Region Estimation

Deep Bayesian Active Learning, A Brief Survey on Recent Advances

Query Efficient Posterior Estimation in Scientific Experiments via Bayesian Active Learning

Dynamic Exploration-Exploitation Trade-Off in Active Learning Regression with Bayesian Hierarchical Modeling

Near-Optimal Bayesian Active Learning with Noisy Observations

Active Learning with Simple Questions

Deep Bayesian Active Learning for Natural Language Processing: Results of a Large-Scale Empirical Study

Online active classification via margin-based and feature-based label queries

Active classification with comparison queries

Bayesian active learning for choice models with deep Gaussian processes

Deep Bayesian Active Learning for Preference Modeling in Large Language Models

Dirichlet Active Learning

Bayesian Active Learning for Discrete Latent Variable Models

Improving Active Learning with a Bayesian Representation of Epistemic Uncertainty

Learning Thresholds with Latent Values and Censored Feedback

Near-Optimal Active Learning of Halfspaces via Query Synthesis in the Noisy Setting

Bayesian Binary Search

Bounded Memory Active Learning through Enriched Queries