Robust Coarse-to-Fine Registration Algorithm for Optical and SAR Images Based on Two Novel Multi-Scale and Multi-Directional Features
Xiaoting Zhang,Yinghua Wang,Jun Liu,Siyuan Wang,Chen Zhang,Hongwei Liu
DOI: https://doi.org/10.1109/tgrs.2024.3417217
IF: 8.2
2024-01-01
IEEE Transactions on Geoscience and Remote Sensing
Abstract:Image registration is the basis for joint utilization of multisource scene information. However, accurate automatic registration of multisource remote sensing images remains a challenging task, especially for optical and synthetic aperture radar (SAR) images. Due to the large geometric and intensity differences between images, many algorithms often fail to accurately register or even mismatch. In this article, we propose a novel coarse-to-fine method with stable high accuracy, which mainly consists of three steps. First, extract consistent features for robust coarse registration. Considering the differences in local gradient magnitudes between optical and SAR images, image intensity preprocessing is performed. Meanwhile, instead of the simple calculations via horizontal and vertical gradient operators, multidirectional gradient operators are defined in two scale spaces to obtain consistent local gradient information. Via multidirectional consistent gradients, highly repeatable keypoints are detected. On the multidirectional gradient maps, support regions with multiple scales are utilized to construct multiscale and multidirectional consistent cross-modal feature descriptors. Second, match the extracted features. Aiming to obtain a reliable alignment in coarse registration step, novel strategies are implemented for the first matching of a cascaded feature matching method. Sufficiently reliable initial matches are established via a new two-way matching strategy, and obvious outliers are removed by exploring the consistencies of both spatial scale and local neighborhood elements of the correct matches. Third, form distinctive pixelwise feature representations for accurate fine registration. In order to increase the distinctiveness of features to distinguish adjacent pixels, a new filter bank based on small receptive fields is defined. Fine features are constructed, appearing as thinner structures at the edges. Meanwhile, through multiscale and multidirectional convolutions, sufficient neighborhood information is mined. Therefore, more precise correspondences can be found to fine-tune the roughly corrected image pair. Overall, a combination method is proposed with a feature-based method and an area-based method for coarse registration and fine registration, respectively. On simulated and real image pairs, the above three steps and the two-stage framework are verified. Experimental results show that the proposed optical-to-SAR image registration method based on the designed multiscale, multidirectional consistent feature and multiscale, multidirectional fine feature (M2F2M) is superior to the current representative feature-based and area-based methods in robustness and accuracy.