Abstract:Guided image restoration (GIR), such as guided depth map super-resolution and pan-sharpening, aims to enhance a target image using guidance information from another image of the same scene. Currently, joint image filtering-inspired deep learning-based methods represent the state-of-the-art for GIR tasks. Those methods either deal with GIR in an end-to-end way by elaborately designing filtering-oriented deep neural network (DNN) modules, focusing on the feature-level fusion of inputs; or explicitly making use of the traditional joint filtering mechanism by parameterizing filtering coefficients with DNNs, working on image-level fusion. The former ones are good at recovering contextual information but tend to lose fine-grained details, while the latter ones can better retain textual information but might lead to content distortions. In this work, to inherit the advantages of both methodologies while mitigating their limitations, we proposed a Simultaneous Feature and Image Guided Fusion (SFIGF) network, that simultaneously considers feature and image-level guided fusion following the guided filter (GF) mechanism. In the feature domain, we connect the cross-attention (CA) with GF, and propose a GF-inspired CA module for better feature-level fusion; in the image domain, we fully explore the GF mechanism and design GF-like structure for better image-level fusion. Since guided fusion is implemented in both feature and image domains, the proposed SFIGF is expected to faithfully reconstruct both contextual and textual information from sources and thus lead to better GIR results. We apply SFIGF to 4 typical GIR tasks, and experimental results on these tasks demonstrate its effectiveness and general availability.

Guided Image Filtering: A Survey and Evaluation Study.

Iterative Self-Guided Image Filtering

Robust Guided Image Filtering

Multichannel guided image filter

Infrared Image Detail Enhancement Based on Guided Filtering with APHE

Enhanced Edge-Perceptual Guided Image Filtering

Deep Attentional Guided Image Filtering

Guided Image Deblurring by Deep Multi-Modal Image Fusion.

Interesting and Impressive: Exploring Design Factors for Product Graphics Interchange Format to Enhance Engagement

GIF2Video: Color Dequantization and Temporal Interpolation of GIF images

A New Image Filtering Method: Nonlocal Image Guided Averaging

Guided Image Restoration via Simultaneous Feature and Image Guided Fusion

Cartoon Image Processing: A Survey

Full-Image Guided Filtering for Fast Stereo Matching

Guided Filtering: Toward Edge-Preserving for Optical Flow

Structure-preserving Guided Retinal Image Filtering and Its Application for Optic Disc Analysis

Fast image dehazing using guided filter

Fine-Grained Image Analysis With Deep Learning: A Survey

TGIF: A New Dataset and Benchmark on Animated GIF Description

Guided image filtering using signal subspace projection.

A Comprehensive Review of Traditional and Deep-Learning-Based Defogging Algorithms