Abstract:In two-view correspondence learning, prevalent multi-layer perceptron (MLP)-based methods struggle with context capturing. To remedy this issue, recent advances innovatively stack convolutional neural network (CNN)-based Resblocks sequentially, showing an inherent proficiency in local context extraction. Yet, such non-issue-specific designs inherit the drawback of CNN’s difficulty in aggregating global context, leading to performance bottlenecks. To address this problem, this prospective study further explores the potential of the CNN-based framework and proposes MC-Net, a top-performing network that integrates both local and global context elegantly and seamlessly. Specifically, considering that sparse motion vectors and a dense motion field can be converted into each other through interpolation and sampling, we first transform unordered matches into image-structured data by estimating the dense motion field implicitly. Then, we design a hierarchical rectifying module to rectify the error of each ordered motion vector with CNN at multiple levels, enabling MC-Net to perceive global context from coarse-level features and local context from fine-level features simultaneously, which facilitates to tackle the discontinuities of the motion field in case of large scene disparity. Finally, we reconstruct comprehensive context-embedded features from rectified motion fields at all levels. Also, instead of using the residuals between rectified and pre-rectified motion vectors at the same layer to reject outliers as in previous studies, which seriously affects the inlier prediction accuracy, we rethink this operation meticulously and modify it to the difference between motion vectors obtained from each layer’s reconstruction and ones from the first layer before transformation, ensuring purer residuals and enhancing the matching performance without extra computational burden. Extensive experiments show that MC-Net outperforms state-of-the-arts on multiple domains and datasets.

Disentangled Non-Local Neural Networks

Unifying Nonlocal Blocks for Neural Networks

Spatially-Aware Context Neural Networks.

Inl: Implicit Non-Local Network

A Local Online Learning Approach for Non-linear Data.

Non-local Neural Networks

Denoised Non-Local Neural Network for Semantic Segmentation

Linear Context Transform Block.

GCNet: Non-Local Networks Meet Squeeze-Excitation Networks and Beyond

A Non-Local Block With Adaptive Regularization Strategy

HSDN: A High-Order Structural Semantic Disentangled Neural Network

Cascaded Non-local Neural Network for Point Cloud Semantic Segmentation

Neural Nearest Neighbors Networks

Learning Context-Based Nonlocal Entropy Modeling for Image Compression

Interpretable Disentanglement of Neural Networks by Extracting Class-Specific Subnetwork

Attention Guided Global Enhancement and Local Refinement Network for Semantic Segmentation

Image Compressed Sensing Using Non-local Neural Network

Non-Local Context Encoder: Robust Biomedical Image Segmentation against Adversarial Attacks

Semantic Segmentation of Aerial Imagery Via Split-Attention Networks with Disentangled Nonlocal and Edge Supervision

MC-Net: Integrating Multi-level Geometric Context for Two-view Correspondence Learning

Non-Local Neural Networks With Grouped Bilinear Attentional Transforms