Abstract:Image matching remains an important and challenging problem in computer vision, especially for the dense correspondence estimation between images with high category-level similarity. The effectiveness of image matching largely depends on the advance of image descriptors. Inspired by the success of Convolutional Neural Network(CNN), we propose a hierarchal image matching method using the CNN feature pyramid, named as CNN Flow. The feature maps output by different layers of CNN tend to encode different information of the input image, such as the semantic information extracted from higher layers and the structural information extracted from lower layers. This nature of CNN feature pyramid is suitable to build the hierarchical image matching framework, which detects the patterns of different levels in an implicit coarse-to-fine manner. In particular, we take advantage of the complementarity of different layers using guidance from higher layer to lower layer. The high-layer features present semantic patterns to cope with the intra-class variations, and the guidance from high layers can resist the semantic ambiguity of low-layer features due to small receptive fields. The bottom-level matching utilize the low-layer features with more structural information to achieve finer matching. On one hand, extensive experiments and analysis demonstrate the superiority of CNN Flow in image dense matching under challenging variations. On the other hand, CNN Flow is demonstrated through various applications, such as fine alignment for intra-class object, scene label transfer and facial expression transfer.

Cycle-Consistency Based Hierarchical Dense Semantic Correspondence

Object-Aware Dense Semantic Correspondence

Deep Semantic Feature Matching Using Confidential Correspondence Consistency

Multi-scale Matching Networks for Semantic Correspondence

Hierarchical Semantic Image Matching Using CNN Feature Pyramid

Consistency Graph Modeling for Semantic Correspondence

Pixel-level Semantic Correspondence Through Layout-aware Representation Learning and Multi-scale Matching Integration

Semantic Correspondence as an Optimal Transport Problem

Learning Semantic Correspondence Exploiting an Object-Level Prior

Semantic-Aware Fine-Grained Correspondence

The Shortest Matching Path Based On Novel Cycle Consistency

Progressive Color Transfer with Dense Semantic Correspondences

Dual-Path Sparse Hierarchical Network for Semantic Segmentation of Remote Sensing Images

Co-saliency Detection Based on Hierarchical Consistency.

Breaking Shortcut: Exploring Fully Convolutional Cycle-Consistency for Video Correspondence Learning

Dense correspondence through descriptor matching

Non-rigid Dense Correspondence with Applications for Image Enhancement

Universal Correspondence Network

Weakly Supervised Learning of Semantic Correspondence Through Cascaded Online Correspondence Refinement

Co-Saliency Detection via Hierarchical Consistency Measure

Anisotropic Multi-Scale Graph Convolutional Network for Dense Shape Correspondence