Abstract:This paper proposes an introspective deep metric learning (IDML) framework for uncertainty-aware comparisons of images. Conventional deep metric learning methods focus on learning a discriminative embedding to describe the semantic features of images, which ignore the existence of uncertainty in each image resulting from noise or semantic ambiguity. Training without awareness of these uncertainties causes the model to overfit the annotated labels during training and produce unsatisfactory judgments during inference. Motivated by this, we argue that a good similarity model should consider the semantic discrepancies with awareness of the uncertainty to better deal with ambiguous images for more robust training. To achieve this, we propose to represent an image using not only a semantic embedding but also an accompanying uncertainty embedding, which describes the semantic characteristics and ambiguity of an image, respectively. We further propose an introspective similarity metric to make similarity judgments between images considering both their semantic differences and ambiguities. The gradient analysis of the proposed metric shows that it enables the model to learn at an adaptive and slower pace to deal with the uncertainty during training. The proposed IDML framework improves the performance of deep metric learning through uncertainty modeling and attains state-of-the-art results on the widely used CUB-200-2011, Cars196, and Stanford Online Products datasets for image retrieval and clustering. We further provide an in-depth analysis of our framework to demonstrate the effectiveness and reliability of IDML. Code: <a class="link-external link-https" href="https://github.com/wzzheng/IDML" rel="external noopener nofollow">this https URL</a>.

Learning IMED Via Shift-Invariant Transformation

The Translation-Invariant Metric and Its Application

On the Translation-Invariance of Image Distance Metric

On the Euclidean Distance of Images

Unaligned Hyperspectral Image Fusion Via Registration and Interpolation Modeling

An Adaptive Image Euclidean Distance

Beyond Iid: Learning To Combine Non-Iid Metrics For Vision Tasks

An Iterative SIFT Based on Intensity and Spatial Information for Remote Sensing Image Registration

Iterative Scale-Invariant Feature Transform for Remote Sensing Image Registration

A Fast Algorithm for Image Euclidean Distance

Social Embedding Image Distance Learning

Distance Metric Learning Via Iterated Support Vector Machines

Distance Metric Learning from Uncertain Side Information with Application to Automated Photo Tagging

Shift-tolerant Perceptual Similarity Metric

EMD Metric Learning

Indescribable Multi-modal Spatial Evaluator

ISImed: A Framework for Self-Supervised Learning using Intrinsic Spatial Information in Medical Images

Image classification with multi-view multi-instance metric learning

Introspective Deep Metric Learning

An Improvement To The Sift Descriptor For Image Representation And Matching

Iterative Metric Learning for Imbalance Data Classification