Abstract:With deep neural network based solution more readily being incorporated in real-world applications, it has been pressing requirement that predictions by such models, especially in safety-critical environments, be highly accurate and well-calibrated. Although some techniques addressing DNN calibration have been proposed, they are only limited to visual classification applications and in-domain predictions. Unfortunately, very little to no attention is paid towards addressing calibration of DNN-based visual object detectors, that occupy similar space and importance in many decision making systems as their visual classification counterparts. In this work, we study the calibration of DNN-based object detection models, particularly under domain shift. To this end, we first propose a new, plug-and-play, train-time calibration loss for object detection (coined as TCD). It can be used with various application-specific loss functions as an auxiliary loss function to improve detection calibration. Second, we devise a new implicit technique for improving calibration in self-training based domain adaptive detectors, featuring a new uncertainty quantification mechanism for object detection. We demonstrate TCD is capable of enhancing calibration with notable margins (1) across different DNN-based object detection paradigms both in in-domain and out-of-domain predictions, and (2) in different domain-adaptive detectors across challenging adaptation scenarios. Finally, we empirically show that our implicit calibration technique can be used in tandem with TCD during adaptation to further boost calibration in diverse domain shift scenarios.

Improve Calibration Robustness of Temperature Scaling by Penalizing Output Entropy

Understanding Negative Calibration from Entropy Perspective

Adaptive temperature scaling for Robust calibration of deep neural networks

Consistency-Guided Temperature Scaling Using Style and Content Information for Out-of-Domain Calibration

Transferable Post-hoc Calibration on Pretrained Transformers in Noisy Text Classification.

A Novel Unsupervised Post-Processing Calibration Method for DNNS with Robustness to Domain Shift

Unsupervised Temperature Scaling: An Unsupervised Post-Processing Calibration Method of Deep Networks

Robust gradient aware and reliable entropy minimization for stable test-time adaptation in dynamic scenarios

Dynamic Batch Norm Statistics Update for Natural Robustness

Benchmarking the Robustness of Spatial-Temporal Models Against Corruptions

Improving Calibration by Relating Focal Loss, Temperature Scaling, and Properness

Towards Improving Calibration in Object Detection Under Domain Shift

Optimizing Calibration by Gaining Aware of Prediction Correctness

Using learned optimizers to make models robust to input noise

Mask-TS Net: Mask Temperature Scaling Uncertainty Calibration for Polyp Segmentation

Impact of Scaled Image on Robustness of Deep Neural Networks

Improving Training and Inference of Face Recognition Models Via Random Temperature Scaling

On Calibration of Modern Quantized Efficient Neural Networks

Robust Calibration with Multi-domain Temperature Scaling

Model Calibration in Dense Classification with Adaptive Label Perturbation

Domain-adaptive and Subgroup-specific Cascaded Temperature Regression for Out-of-distribution Calibration