Abstract:Optical and synthetic aperture radar (SAR) image keypoint detection is an important foundation for multimodal remote sensing image matching. The influence of nonlinear radiometric differences and geometric deformation between optical and SAR images leads to low repeatability of existing keypoint detection methods. To address the problem that existing keypoint detection methods cannot provide the required homonymous points for heterogeneous image matching, we propose a keypoint detection method [multilevel attention Siamese network for keypoint detection in optical and SAR images (SKD-Net)] and improve it in terms of both network structure and network optimization. First, we propose a multilevel attention Siamese network, which is composed of multiple convolutional modules and transformer modules with shared weights to extract common features at different levels for keypoint detection. We introduce a transformer module in the keypoint detection pipeline and fuse shallow and deep features to obtain more spatial and rich semantic information to facilitate heterogeneous image keypoint detection. Then, to ensure that the detected keypoints have more homonymous points and localization accuracy, we propose a position-consistent loss. Unlike previous loss functions, our designed position-consistent loss function takes the differences between heterogeneous image score maps into account, and it autonomously selects the optimized correct point pairs to enable the network to perform correct learning. Finally, extensive experiments show that our detection method outperforms the current state-of-the-art keypoint detection methods in terms of repeatability, localization accuracy, and matching performance. Our source code is available at https://github.com/zhangschen/SKD-Net.

Key.Net: Keypoint Detection by Handcrafted and Learned CNN Filters Revisited

DKNAS: A Practical Deep Keypoint Extraction Framework Based on Neural Architecture Search

Sketch-R2CNN: an Attentive Network for Vector Sketch Recognition

CoKe: Localized Contrastive Learning for Robust Keypoint Detection

Learning to Make Keypoints Sub-Pixel Accurate

Learning Higher-order Object Interactions for Keypoint-based Video Understanding

Dynamic Keypoint Detection Network for Image Matching

SiLK -- Simple Learned Keypoints

KeypointDETR: an End-to-End 3D Keypoint Detector

A Scale and Rotational Invariant Key-point Detector based on Sparse Coding

Integrated Deconvolution Keypoint Detector and Descriptor Network

Multilevel Attention Siamese Network for Keypoint Detection in Optical and SAR Images

A Coarse-Fine Network for Keypoint Localization

Multi-person pose estimation using atrous convolution

Reinforced Feature Points: Optimizing Feature Detection and Description for a High-Level Task

CenterNet: Keypoint Triplets for Object Detection

SK-Net: Deep Learning on Point Cloud via End-to-end Discovery of Spatial Keypoints

Towards High Performance Human Keypoint Detection

Novel Scale-Invariant Keypoint Detector

Attend and Guide (AG-Net): A Keypoints-driven Attention-based Deep Network for Image Recognition