Deep Probabilistic Ensembles: Approximate Variational Inference through KL Regularization

Kashyap Chitta,Jose M. Alvarez,Adam Lesnikowski
DOI: https://doi.org/10.48550/arXiv.1811.02640
IF: 5.414
2018-11-06
Machine Learning
Abstract:In this paper, we introduce Deep Probabilistic Ensembles (DPEs), a scalable technique that uses a regularized ensemble to approximate a deep Bayesian Neural Network (BNN). We do so by incorporating a KL divergence penalty term into the training objective of an ensemble, derived from the evidence lower bound used in variational inference. We evaluate the uncertainty estimates obtained from our models for active learning on visual classification. Our approach steadily improves upon active learning baselines as the annotation budget is increased.
What problem does this paper attempt to address?