Abstract:Distributed deep learning has recently been attracting more attention in remote sensing (RS) applications due to the challenges posed by the increased amount of open data that are produced daily by Earth observation programs. However, the high communication costs of sending model updates among multiple nodes are a significant bottleneck for scalable distributed learning. Gradient sparsification has been validated as an effective gradient compression (GC) technique for reducing communication costs and thus accelerating the training speed. Existing state-of-the-art gradient sparsification methods are mostly based on the "larger-absolute-more-important" criterion, ignoring the importance of small gradients, which is generally observed to affect the performance. Inspired by informative representation of manifold structures from neighborhood information, we propose a simple yet effective dynamic GC scheme leveraging neighborhood statistics indicator for RS image interpretation, termed RS-DGC. We first enhance the interdependence between gradients by introducing the gradient neighborhood to reduce the effect of random noise. The key component of RS-DGC is a neighborhood statistical indicator (NSI), which can quantify the importance of gradients within a specified neighborhood on each node to sparsify the local gradients before gradient transmission in each iteration. Further, a layer-wise dynamic compression scheme is proposed to track the important changes of each layer in real time. Extensive downstream tasks validate the superiority of our method in terms of intelligent interpretation of RS images. For example, we achieve an accuracy improvement of 0.51% with more than communication compression on the NWPU-RESISC45 dataset using VGG-19 network. To the best of our knowledge, this is the first GC method designed for RS images and downstream tasks, achieving a successful trade-off between high compression ratio and performance.

Riemannian Gradient Descent Method to Joint Blind Super-Resolution and Demixing in ISAC

Simultaneous Blind Demixing and Super-resolution via Vectorized Hankel Lift

Fast and Provable Simultaneous Blind Super-Resolution and Demixing for Point Source Signals: Scaled Gradient Descent without Regularization

Joint Demosaicing and Denoising with Gradient Guidance in Quad Bayer CFA

Remote Sensing Single-Image Resolution Improvement Using A Deep Gradient-Aware Network with Image-Specific Enhancement

Simpler Gradient Methods for Blind Super-Resolution with Lower Iteration Complexity

Jointly RS Image Deblurring and Super-Resolution with Adjustable-Kernel and Multi-Domain Attention

Hyperspectral Image Reconstruction of SD-CASSI Based on Nonlocal Low-Rank Tensor Prior

RS-DGC: Exploring Neighborhood Statistics for Dynamic Gradient Compression on Remote Sensing Image Interpretation

Deep Learning for Super-Resolution Channel Estimation in Reconfigurable Intelligent Surface Aided Systems.

ISAC Super-Resolution Receiver via Lifted Atomic Norm Minimization

Supervised Descent Method for Solving Nonlinear Least Squares Problems in Computer Vision

Image Super-Resolution Based on Adaptive Joint Distribution Modeling

Joint Localization and Information Transfer for RIS Aided Full-Duplex Systems.

Global Convergence of a Grassmannian Gradient Descent Algorithm for Subspace Estimation

How Does the Low-Rank Matrix Decomposition Help Internal and External Learnings for Super-Resolution

Super-Resolution Compressed Sensing: A Generalized Iterative Reweighted L2 Approach.

Joint Demosaicking And Blind Deblurring Using Deep Convolutional Neural Network

Joint Demosaicing and Denoising With Self Guidance

LRSD-ADMM-NET: Simultaneous Super- Resolution Imaging and Target Detection for Forward-Looking Scanning Radar

Super-Resolution Compressed Sensing: an Iterative Reweighted Algorithm for Joint Parameter Learning and Sparse Signal Recovery