Integrating SIFT and CNN Feature Matching for Partial-Duplicate Image Detection

Zhili Zhou,Q. M. Jonathan Wu,Shaohua Wan,Wendi Sun,Xingming Sun

DOI: https://doi.org/10.1109/tetci.2019.2909936

2020-10-01

IEEE Transactions on Emerging Topics in Computational Intelligence

Abstract:With the increasing popularity of various deep neural networks in the area of computational intelligence, the research attention for content-based image detectionretrieval has been shifted from the handcrafted local features such as scale invariant feature transform (SIFT) to the features derived from convolutional neural networks (CNN). However, the existing image-based CNN features, directly extracted from the entire images, are not suitable for detecting small duplicate regions, while region-based CNN features show limited robustness to a variety of image modifications such as rescaling, occlusion, and noise adding. These will affect the performance of partial-duplicate image detection. To address these issues, we propose an integrated feature matching scheme, which integrates the matching of SIFT features and CNN features between images for partial-duplicate image detection. In this scheme, we first implement SIFT feature matching based on the bag-of-visual-words model to detect the potential duplicate region pairs between images, and then match the CNN features of these regions extracted from the deep convolutional layer of CNN to compute image similarity. Since both the good robustness of SIFT features and the high discriminative power of CNN features are sufficiently explored, our scheme allows an accurate detection. Experimental results show that the proposed approach provides superior accuracy than the state of the arts, while achieves comparable efficiency.