Abstract:Image-based environment perception is an important component especially for driver assistance systems or autonomous driving. In this scope, modern neuronal networks are used to identify multiple objects as well as the according position and size information within a single frame. The performance of such an object detection model is important for the overall performance of the whole system. However, a detection model might also predict these objects under a certain degree of uncertainty. [...] In this work, we examine the semantic uncertainty (which object type?) as well as the spatial uncertainty (where is the object and how large is it?). We evaluate if the predicted uncertainties of an object detection model match with the observed error that is achieved on real-world data. In the first part of this work, we introduce the definition for confidence calibration of the semantic uncertainty in the context of object detection, instance segmentation, and semantic segmentation. We integrate additional position information in our examinations to evaluate the effect of the object's position on the semantic calibration properties. Besides measuring calibration, it is also possible to perform a post-hoc recalibration of semantic uncertainty that might have turned out to be miscalibrated. [...] The second part of this work deals with the spatial uncertainty obtained by a probabilistic detection model. [...] We review and extend common calibration methods so that it is possible to obtain parametric uncertainty distributions for the position information in a more flexible way. In the last part, we demonstrate a possible use-case for our derived calibration methods in the context of object tracking. [...] We integrate our previously proposed calibration techniques and demonstrate the usefulness of semantic and spatial uncertainty calibration in a subsequent process. [...]

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the uncertainty calibration problem of object detection models in image - based environmental perception, especially in autonomous driving or driving assistance systems. Specifically, the paper focuses on how to evaluate and improve the uncertainty estimation of object detection models when predicting object types (semantic uncertainty) and their positions and sizes (spatial uncertainty). ### Main Problems 1. **Semantic Uncertainty Calibration**: - The paper explores how to evaluate whether the semantic uncertainty (i.e., object type) predicted by the object detection model is consistent with the actual error. - It proposes new methods for location - dependent semantic uncertainty calibration and introduces the concept of Bayesian confidence calibration to determine the epistemic uncertainty inherent in the calibration method itself. 2. **Spatial Uncertainty Calibration**: - The paper examines multiple definitions of regression uncertainty calibration and extends common calibration methods to more flexibly obtain the parametric uncertainty distribution of location information. - It verifies the effectiveness of these methods through experiments and shows how to apply the calibrated spatial uncertainty to object tracking tasks, thereby significantly improving the performance of object tracking. ### Background - **Importance of Environmental Perception**: In autonomous driving and driving assistance systems, image - based environmental perception is an important component. Modern neural networks are used to identify multiple objects in a single frame and their position and size information. - **Types of Uncertainty**: The paper distinguishes between epistemic model uncertainty (e.g., due to insufficient knowledge) and data uncertainty (e.g., due to random effects in the input data). These uncertainties may have a significant impact on the safety of the system. - **Existing Challenges**: Common object detection models often have the problem of inaccurate calibration, which includes not only semantic uncertainty but also spatial uncertainty. This inaccuracy may lead to errors in the system during critical decision - making. ### Solutions - **Semantic Uncertainty Calibration**: - It introduces the definition of confidence calibration and proposes a multivariate confidence calibration method. - It develops a location - dependent calibration method and verifies its effectiveness through experiments. - **Spatial Uncertainty Calibration**: - It examines and extends existing regression uncertainty calibration methods. - It proposes a method for parameter calibration using Gaussian processes and verifies its performance through experiments. ### Applications - **Object Tracking**: The paper shows how to apply the calibration methods to object tracking tasks. By calibrating semantic and spatial uncertainties, it significantly improves the performance of object tracking. ### Conclusions - The paper shows that common object detection models often have the problem of inaccurate calibration. Through the proposed new calibration methods, these inaccurate uncertainty estimates can be effectively corrected, thereby making a valuable contribution to the overall environmental perception process.

Uncertainty Calibration and its Application to Object Detection

Parametric and Multivariate Uncertainty Calibration for Regression and Object Detection

A Theoretical and Practical Framework for Evaluating Uncertainty Calibration in Object Detection

Multivariate Confidence Calibration for Object Detection

Confidence Calibration for Object Detection and Segmentation

On the Calibration of Uncertainty Estimation in LiDAR-based Semantic Segmentation

Calibrated Perception Uncertainty Across Objects and Regions in Bird's-Eye-View

CertainNet: Sampling-free Uncertainty Estimation for Object Detection

Inferring Bias and Uncertainty in Camera Calibration

An Uncertainty Estimation Framework for Probabilistic Object Detection

Overcoming the Limitations of Localization Uncertainty: Efficient & Exact Non-Linear Post-Processing and Calibration

Uncertainty Estimation for 3D Object Detection via Evidential Learning

From Black-box to White-box: Examining Confidence Calibration under different Conditions

Evaluating and Calibrating Uncertainty Prediction in Regression Tasks

Unified Uncertainty Calibration

Calibrating Ensembles for Scalable Uncertainty Quantification in Deep Learning-based Medical Segmentation

Incorporating Data Uncertainty in Object Tracking Algorithms

Labels Are Not Perfect: Inferring Spatial Uncertainty in Object Detection

Towards Building Self-Aware Object Detectors via Reliable Uncertainty Quantification and Calibration

On Calibration of Object Detectors: Pitfalls, Evaluation and Baselines

Uncertainty-Aware Deep Calibrated Salient Object Detection