Abstract:Most existing image-text matching methods adopt triplet loss as the optimization objective, and choosing a proper negative sample for the triplet of <anchor, positive, negative> is important for effectively training the model, e.g., hard negatives make the model learn efficiently and effectively. However, we observe that existing methods mainly employ the most similar samples as hard negatives, which may not be true negatives. In other words, the samples with high similarity but not paired with the anchor may reserve positive semantic associations, and we call them false negatives. Repelling these false negatives in triplet loss would mislead the semantic representation learning and result in inferior retrieval performance. In this paper, we propose a novel False Negative Elimination (FNE) strategy to select negatives via sampling, which could alleviate the problem introduced by false negatives. Specifically, we first construct the distributions of positive and negative samples separately via their similarities with the anchor, based on the features extracted from image and text encoders. Then we calculate the false negative probability of a given sample based on its similarity with the anchor and the above distributions via the Bayes' rule, which is employed as the sampling weight during negative sampling process. Since there may not exist any false negative in a small batch size, we design a memory module with momentum to retain a large negative buffer and implement our negative sampling strategy spanning over the buffer. In addition, to make the model focus on hard negatives, we reassign the sampling weights for the simple negatives with a cut-down strategy. The extensive experiments are conducted on Flickr30K and MS-COCO, and the results demonstrate the superiority of our proposed false negative elimination strategy. The code is available at <a class="link-external link-https" href="https://github.com/LuminosityX/FNE" rel="external noopener nofollow">this https URL</a>.

Open-Set Face Verification Algorithm Using Competitive Negative Samples

USING COMPETITIVE PROTOTYPES FOR THE VERIFICATION ON UNSPECIFIC PERSON

Algorithm of face anti-spoofing based on pseudo-negative features generation

Robust 3D Face Recognition by Local Shape Difference Boosting

Toward Open-Set Face Recognition

Performance Evaluation System in Open Set Face Recognition

Open-Set Biometrics: Beyond Good Closed-Set Models

Your Negative May not Be True Negative: Boosting Image-Text Matching with False Negative Elimination

Computers do better than experts matching faces in a large population

2 Face Authentication Test on the BANCA Database

A Novel High-Performance Face Anti-Spoofing Detection Method

Fusing Two Kinds of Virtual Samples for Small Sample Face Recognition

Face Verification System Based on Texture Feature with Single Sample Per Person

Face Recognition under Occlusions and Variant Expressions with Partial Similarity

Efficient Verification-Based Face Identification

A New Face Recognition Method Based on SVD Perturbation for Single Example Image Per Person

Discriminative Prototype Learning in Open Set Face Recognition

Negative facial expression recognition based on improved convolutional neural networks

Using known nonself samples to improve negative selection algorithm

A New Algorithm for Generalized Optimal Discriminant Vectors

Comparison between Two Kinds of PCA-Based Face Recognition Algorithms by Sign Hypothesis Testing Strategy