Abstract:Accurately estimating image quality and model robustness improvement are critical challenges in unconstrained face recognition, which can be addressed through uncertainty estimation via probabilistic face embeddings. Previous research mainly focused on uncertainty estimation in face verification, leaving the open-set face recognition task underexplored. In open-set face recognition, one seeks to classify an image, which could also be unknown. Here, the low variance of probabilistic embedding does not imply a low error probability: an image embedding could be close to several classes in a gallery, thus yielding high uncertainty. We propose a method aware of two sources of ambiguity in the open-set recognition system: (1) the gallery uncertainty caused by overlapping classes and (2) the uncertainty of the face embeddings. To detect both types, we use a Bayesian probabilistic model of embedding distribution, which provides a principled uncertainty estimate. Challenging open-set face recognition datasets, such as IJB-C, serve as a testbed for our method. We also propose a new open-set recognition protocol for whale and dolphin identification. The proposed approach better identifies recognition errors than uncertainty estimation methods based solely on image quality.

What problem does this paper attempt to address?

### Problems Addressed by the Paper This paper aims to address the issue of uncertainty estimation in Open-Set Face Recognition (OSFR). Specifically, the authors focus on how to effectively estimate image quality and model robustness in open-set face recognition tasks to improve the system's recognition accuracy and reliability. ### Background and Challenges 1. **Insufficiency of Existing Research**: - Existing research mainly focuses on uncertainty estimation in face verification tasks, with less exploration in open-set face recognition tasks. - In open-set face recognition, test images may be unknown, increasing the complexity and uncertainty of recognition. 2. **Sources of Uncertainty**: - **Gallery Uncertainty**: Due to possible overlap in gallery categories, the embedding vector of the test image may be close to multiple categories, resulting in high uncertainty. - **Facial Embedding Uncertainty**: Quality issues of the input image may lead to uncertainty in the embedding vector. 3. **Limitations of Existing Methods**: - Direct application of existing uncertainty estimation methods lacks the necessary attributes to work effectively in open-set face recognition. - For example, when the embedding vector of the test image is close to multiple categories, existing uncertainty estimation methods may not correctly reflect the ambiguity of recognition. ### Proposed Method 1. **Bayesian Probabilistic Model**: - The authors propose a Bayesian probabilistic model to estimate the uncertainty of embedding distributions. - This model naturally combines two sources of uncertainty: gallery uncertainty and facial embedding uncertainty. 2. **Improved Decision Rule**: - By reformulating the open-set face recognition problem as a classification problem, a new "out-of-gallery" category corresponding to unknown subjects is introduced. - This allows the system to more accurately recognize and reject unknown test samples. 3. **Uncertainty Estimation**: - The von Mises-Fisher (vMF) and Power distributions are used to model the conditional embedding distribution of each gallery category. - By using Laplace approximation, the embedding distribution and category distribution are combined to obtain the final uncertainty estimation. 4. **Calibration**: - To ensure the comparability of uncertainty estimates from different sources, the authors use beta calibration and logistic calibration methods. ### Experiments and Evaluation 1. **Datasets**: - Experiments are conducted using the IJB-C, IJB-B, and Happywhale datasets. - These datasets are widely used in face recognition research. 2. **Evaluation Metrics**: - Recognition performance is evaluated using F1 score, Detection and Identification Rate (DIR), and False Alarm Rate (FAR). - The quality of uncertainty estimation is evaluated using the area under the Predictive Rejection Curve (AUC-PR). 3. **Experimental Results**: - The experimental results show that the proposed method outperforms image quality-based uncertainty estimation methods and simple heuristic methods in both recognition performance and uncertainty estimation. ### Main Contributions 1. **Gallery-Aware Probabilistic Method**: - A gallery-aware probabilistic method is proposed, capable of generating calibrated decision probabilities and naturally incorporating the relative positional information of facial embeddings. 2. **Multi-Source Uncertainty Estimation**: - A method combining multiple uncertainty estimations is proposed, including gallery-aware uncertainty measures and neural network-based facial image quality estimators. 3. **New Open-Set Recognition Protocol**: - A new open-set recognition protocol for the Happywhale dataset is proposed, validating the method's applicability in different domains. 4. **Experimental Evaluation**: - Through detailed experimental evaluation, the superiority of the proposed method in recognition performance and uncertainty estimation is demonstrated. In summary, this paper effectively addresses the issue of uncertainty estimation in open-set face recognition by introducing a new Bayesian probabilistic model and improved decision rules, enhancing the system's recognition accuracy and reliability.

Gallery-Aware Uncertainty Estimation For Open-Set Face Recognition

Uncertainty-Based Active Learning Via Sparse Modeling for Image Classification

Exploring Aleatoric Uncertainty in Object Detection via Vision Foundation Models

Data Uncertainty in Face Recognition

Uncertainty Modeling for Gaze Estimation.

Fast and reliable probabilistic face embeddings based on constrained data uncertainty estimation

Open-set Face Recognition for Small Galleries Using Siamese Networks

Toward Open-Set Face Recognition

Open-set Face Recognition using Ensembles trained on Clustered Data

Uncertainty-Aware Image Captioning

On the Estimation of Image-matching Uncertainty in Visual Place Recognition

An Uncertainty Estimation Framework for Probabilistic Object Detection

UNCERTAINTY ESTIMATION FOR END-TO-END LEARNED DENSE STEREO MATCHING VIA PROBABILISTIC DEEP LEARNING

What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?

Enhancing Facial Expression Recognition Under Data Uncertainty Based on Embedding Proximity

Generalizing Face Forgery Detection Via Uncertainty Learning

Uncertainty for Identifying Open-Set Errors in Visual Object Detection

Probabilistic Elastic Part Model for Real-World Face Recognition.

Evaluating Uncertainty Calibration for Open-Set Recognition

Mid-level Representation Enhancement and Graph Embedded Uncertainty Suppressing for Facial Expression Recognition

Uncertainty-oriented Order Learning for Facial Beauty Prediction