Globally Correlation-Aware Hard Negative Generation

Wenjie Peng,Hongxiang Huang,Tianshui Chen,Quhui Ke,Gang Dai,Shuangping Huang

2024-11-20

Abstract:Hard negative generation aims to generate informative negative samples that help to determine the decision boundaries and thus facilitate advancing deep metric learning. Current works select pair/triplet samples, learn their correlations, and fuse them to generate hard negatives. However, these works merely consider the local correlations of selected samples, ignoring global sample correlations that would provide more significant information to generate more informative negatives. In this work, we propose a Globally Correlation-Aware Hard Negative Generation (GCA-HNG) framework, which first learns sample correlations from a global perspective and exploits these correlations to guide generating hardness-adaptive and diverse negatives. Specifically, this approach begins by constructing a structured graph to model sample correlations, where each node represents a specific sample and each edge represents the correlations between corresponding samples. Then, we introduce an iterative graph message propagation to propagate the messages of node and edge through the whole graph and thus learn the sample correlations globally. Finally, with the guidance of the learned global correlations, we propose a channel-adaptive manner to combine an anchor and multiple negatives for HNG. Compared to current methods, GCA-HNG allows perceiving sample correlations with numerous negatives from a global and comprehensive perspective and generates the negatives with better hardness and diversity. Extensive experiment results demonstrate that the proposed GCA-HNG is superior to related methods on four image retrieval benchmark datasets. Codes and trained models are available at \url{<a class="link-external link-https" href="https://github.com/PWenJay/GCA-HNG" rel="external noopener nofollow">this https URL</a>}.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to generate more informative hard negative samples in deep metric learning. Current methods usually select sample pairs or triplets, learn the local correlations among them, and generate hard negative samples through these correlations. However, these methods only consider the local correlations among the selected samples and ignore the global sample correlations, which can provide more information to generate more valuable negative samples. Therefore, the paper proposes a new framework - Globally Correlation - Aware Hard Negative Generation (GCA - HNG), aiming to learn the correlations of samples from a global perspective and use these correlations to guide the generation of highly adaptable and diverse hard negative samples, so as to promote the development of deep metric learning.

Specifically, the GCA - HNG framework first constructs a structured graph to model sample correlations, where each node represents a specific sample and each edge represents the correlation between the corresponding samples. Then, an iterative graph message - propagation mechanism is introduced to pass messages of nodes and edges throughout the graph, thereby learning sample correlations from a global perspective. Finally, under the guidance of the learned global correlations, a channel - adaptive method is proposed to combine the anchor and multiple negative samples for hard negative sample generation (HNG). Compared with existing methods, GCA - HNG allows to perceive the correlations among a large number of negative samples from a broader and more comprehensive perspective and generate negative samples with higher hardness and stronger diversity. Experimental results show that GCA - HNG outperforms related methods on four image retrieval benchmark datasets.