Abstract:The reconstruction of missing pixels is essential for remote sensing images, as they often suffer from problems such as covering, dead pixels, and scan line corrector (SLC)-off. Image inpainting techniques can solve these problems, as they can generate realistic content for the unknown regions of an image based on the known regions. Recently, convolutional neural network (CNN)-based inpainting methods have integrated the attention mechanism to improve inpainting performance, as they can capture long-range dependencies and adapt to inputs in a flexible manner. However, to obtain the attention map for each feature, they compute the similarities between the feature and the entire feature map, which may introduce noise from irrelevant features. To address this problem, we propose a novel adaptive attention (Ada-attention) that uses an offset position subnet to adaptively select the most relevant keys and values based on self-attention. This enables the attention to be focused on essential features and model more informative dependencies on the global range. Ada-attention first employs an offset subnet to predict offset position maps on the query feature map; then, it samples the most relevant features from the input feature map based on the offset position; next, it computes key and value maps for self-attention using the sampled features; finally, using the query, key and value maps, the self-attention outputs the reconstructed feature map. Based on Ada-attention, we customized a u-shaped adaptive-attention completing network (AACNet) to reconstruct missing regions. Experimental results on several digital remote sensing and natural image datasets, using two image inpainting models and two remote sensing image reconstruction approaches, demonstrate that the proposed AACNet achieves a good quantitative performance and good visual restoration results with regard to object integrity, texture/edge detail, and structural consistency. Ablation studies indicate that Ada-attention outperforms self-attention in terms of PSNR by 0.66%, SSIM by 0.74%, and MAE by 3.9%, and can focus on valuable global features using the adaptive offset subnet. Additionally, our approach has also been successfully applied to remove real clouds in remote sensing images, generating credible content for cloudy regions.

Adaptive Quasi-Newton Algorithm for Remote Sensing

A lightweight and stochastic depth residual attention network for remote sensing scene classification

An Adaptive Remote Stochastic Gradient Method for Training Neural Networks

Nonlocal Spatial–Spectral Neural Network for Hyperspectral Image Denoising

Deep Amended Gradient Descent for Efficient Spectral Reconstruction from Single RGB Images

Improving the Accuracy of Binarized Neural Networks and Application on Remote Sensing Data

Remote Sensing Image Classification Based on Neural Networks Designed Using an Efficient Neural Architecture Search Methodology

Simple Yet Effective Fine-Tuning of Deep CNNs Using an Auxiliary Classification Loss for Remote Sensing Scene Classification

Brain-Inspired Online Adaptation for Remote Sensing with Spiking Neural Network

Asynchronous Parallel Stochastic Quasi-Newton Methods

Adaptive Multiscale Deep Fusion Residual Network for Remote Sensing Image Classification

A Pseudoinverse Incremental Algorithm For Fast Training Deep Neural Networks With Application To Spectra Pattern Recognition

Basis Pursuit Denoising via Recurrent Neural Network Applied to Super-resolving SAR Tomography

Parameterization of point-cloud freeform surfaces using adaptive sequential learning RBFnetworks

LRR-Net: An Interpretable Deep Unfolding Network for Hyperspectral Anomaly Detection

Adaptive-Attention Completing Network for Remote Sensing Image

Modified quasi-Newton algorithm for training large-scale feedforward neural networks and its application

QuadraNet V2: Efficient and Sustainable Training of High-Order Neural Networks with Quadratic Adaptation

Lightweight Feedback Convolution Neural Network for Remote Sensing Images Super-Resolution

Asymmetric Dual-Direction Quasi-Recursive Network for Single Hyperspectral Image Super-Resolution

FST-EfficientNetV2: Exceptional Image Classification for Remote Sensing