Abstract:We present a general learning-based solution for restoring images suffering from spatially-varying degradations. Prior approaches are typically degradation-specific and employ the same processing across different images and different pixels within. However, we hypothesize that such spatially rigid processing is suboptimal for simultaneously restoring the degraded pixels as well as reconstructing the clean regions of the image. To overcome this limitation, we propose SPAIR, a network design that harnesses distortion-localization information and dynamically adjusts computation to difficult regions in the image. SPAIR comprises of two components, (1) a localization network that identifies degraded pixels, and (2) a restoration network that exploits knowledge from the localization network in filter and feature domain to selectively and adaptively restore degraded pixels. Our key idea is to exploit the non-uniformity of heavy degradations in spatial-domain and suitably embed this knowledge within distortion-guided modules performing sparse normalization, feature extraction and attention. Our architecture is agnostic to physical formation model and generalizes across several types of spatially-varying degradations. We demonstrate the efficacy of SPAIR individually on four restoration tasks-removal of rain-streaks, raindrops, shadows and motion blur. Extensive qualitative and quantitative comparisons with prior art on 11 benchmark datasets demonstrate that our degradation-agnostic network design offers significant performance gains over state-of-the-art degradation-specific architectures. Code available at <a class="link-external link-https" href="https://github.com/human-analysis/spatially-adaptive-image-restoration" rel="external noopener nofollow">this https URL</a>.

PSTAIC regularization for 2D spatiotemporal image reconstruction

STAIC regularization for spatio-temporal image reconstruction

Hyperspectral Restoration Via $L_0$ Gradient Regularized Low-Rank Tensor Factorization

Image Restoration by Combined Order Regularization with Optimal Spatial Adaptation

Photo-acoustic tomographic image reconstruction from reduced data using physically inspired regularization

Spatially-Adaptive Image Restoration using Distortion-Guided Networks

Hyperspectral Image Restoration via Spatial-Spectral Residual Total Variation Regularized Low-Rank Tensor Decomposition

Hyperspectral Image Restoration Using Weighted Group Sparsity-Regularized Low-Rank Tensor Decomposition

Reweighted Anisotropic Total Variation Minimization for Limited-Angle CT Reconstruction

Learned Regularization for Image Reconstruction in Sparse-View Photoacoustic Tomography

Single Image Restoration Through ℓ2-Relaxed Truncated ℓ0 Analysis-Based Sparse Optimization in Tight Frames

Single image restoration through l 2 -relaxed truncated l 0 analysis-based sparse optimization in tight frames

Weighted Group Sparsity Regularized Low-Rank Tensor Decomposition for Hyperspectral Image Restoration.

Real-Time Limited-View CT Inpainting and Reconstruction with Dual Domain Based on Spatial Information

Spectral Quadratic Variation Regularized Auto-Weighted Tensor Ring Decomposition for Hyperspectral Image Reconstruction

Spectral Quadratic Variation Regularized Autoweighted Tensor Ring Decomposition for Hyperspectral Image Reconstruction

Mumford-Shah and Potts Regularization for Manifold-Valued Data with Applications to DTI and Q-Ball Imaging

Regularization with optimal space-time priors

Hyperspectral Image Restoration Using 3-D Hybrid Higher Degree Total Variation Regularized Nonconvex Local Low-Rank Tensor Recovery

Image restoration via the adaptive T V p regularization

Hyperspectral Image Restoration via Total Variation Regularized Low-rank Tensor Decomposition