Abstract:Recent innovations in training deep convolutional neural network (ConvNet) models have motivated the design of new methods to automatically learn local image descriptors. The latest deep ConvNets proposed for this task consist of a siamese network that is trained by penalising misclassification of pairs of local image patches. Current results from machine learning show that replacing this siamese by a triplet network can improve the classification accuracy in several problems, but this has yet to be demonstrated for local image descriptor learning. Moreover, current siamese and triplet networks have been trained with stochastic gradient descent that computes the gradient from individual pairs or triplets of local image patches, which can make them prone to overfitting. In this paper, we first propose the use of triplet networks for the problem of local image descriptor learning. Furthermore, we also propose the use of a global loss that minimises the overall classification error in the training set, which can improve the generalisation capability of the model. Using the UBC benchmark dataset for comparing local image descriptors, we show that the triplet network produces a more accurate embedding than the siamese network in terms of the UBC dataset errors. Moreover, we also demonstrate that a combination of the triplet and global losses produces the best embedding in the field, using this triplet network. Finally, we also show that the use of the central-surround siamese network trained with the global loss produces the best result of the field on the UBC dataset. Pre-trained models are available online at <a class="link-external link-https" href="https://github.com/vijaykbg/deep-patchmatch" rel="external noopener nofollow">this https URL</a>

Image similarity using Deep CNN and Curriculum Learning

Visualizing Deep Similarity Networks

Deep Unsupervised Learning of Visual Similarities

Deep Image Similarity Measurement Based on the Improved Triplet Network with Spatial Pyramid Pooling.

Neural Similarity Learning

Deep Image Category Discovery Using a Transferred Similarity Function

Similarity-based Text Recognition by Deeply Supervised Siamese Network

A Time Sequence Images Matching Method Based on the Siamese Network

Learning visual similarity for product design with convolutional neural networks

Deep Siamese Multi-level Feature Network for VIS and NIR Image Matching

Deep Convolutional Neural Network Using Triplet Loss to Distinguish the Identical Twins.

Siamese coding network and pair similarity prediction for near-duplicate image detection

Learning Local Image Descriptors with Deep Siamese and Triplet Convolutional Networks by Minimising Global Loss Functions

DeepSIM: a novel deep learning method for graph similarity computation

Vehicle Verification Based on Deep Siamese Network with Similarity Metric

Learning Preference-Based Similarities from Face Images using Siamese Multi-Task CNNs

Optical and SAR Image Matching Using Pixelwise Deep Dense Features

DualNet: Learn Complementary Features for Image Recognition

Universal Correspondence Network

Similarity-Based Unsupervised Deep Transfer Learning for Remote Sensing Image Retrieval

Deep Sparse Coding Network for Image Classification