PAC Confidence Sets for Deep Neural Networks via Calibrated Prediction

Sangdon Park,Osbert Bastani,Nikolai Matni,Insup Lee
DOI: https://doi.org/10.48550/arXiv.2001.00106
2020-02-16
Abstract:We propose an algorithm combining calibrated prediction and generalization bounds from learning theory to construct confidence sets for deep neural networks with PAC guarantees---i.e., the confidence set for a given input contains the true label with high probability. We demonstrate how our approach can be used to construct PAC confidence sets on ResNet for ImageNet, a visual object tracking model, and a dynamics model for the half-cheetah reinforcement learning problem.
Machine Learning
What problem does this paper attempt to address?