Abstract:Super-resolution (SR) and image generation are important tasks in computer vision and are widely adopted in real-world applications. Most existing methods, however, generate images only at fixed-scale magnification and suffer from over-smoothing and artifacts. Additionally, they do not offer enough diversity of output images nor image consistency at different scales. Most relevant work applied Implicit Neural Representation (INR) to the denoising diffusion model to obtain continuous-resolution yet diverse and high-quality SR results. Since this model operates in the image space, the larger the resolution of image is produced, the more memory and inference time is required, and it also does not maintain scale-specific consistency. We propose a novel pipeline that can super-resolve an input image or generate from a random noise a novel image at arbitrary scales. The method consists of a pretrained auto-encoder, a latent diffusion model, and an implicit neural decoder, and their learning strategies. The proposed method adopts diffusion processes in a latent space, thus efficient, yet aligned with output image space decoded by MLPs at arbitrary scales. More specifically, our arbitrary-scale decoder is designed by the symmetric decoder w/o up-scaling from the pretrained auto-encoder, and Local Implicit Image Function (LIIF) in series. The latent diffusion process is learnt by the denoising and the alignment losses jointly. Errors in output images are backpropagated via the fixed decoder, improving the quality of output images. In the extensive experiments using multiple public benchmarks on the two tasks i.e. image super-resolution and novel image generation at arbitrary scales, the proposed method outperforms relevant methods in metrics of image quality, diversity and scale consistency. It is significantly better than the relevant prior-art in the inference speed and memory usage.

Deformable CNN with Position Encoding for Arbitrary-Scale Super-Resolution

Epistemic-Uncertainty-Based Divide-and-Conquer Network for Single-Image Super-Resolution

Enhanced Implicit Function-Based Network for Arbitrary-Scale Image Super-Resolution

UltraSR: Spatial Encoding is a Missing Key for Implicit Image Function-based Arbitrary-Scale Super-Resolution

Orientation-Aware Deep Neural Network for Real Image Super-Resolution.

Deformable and residual convolutional network for image super-resolution

Image Superresolution using Scale-Recurrent Dense Network

Enhancing Multi-Scale Implicit Learning in Image Super-Resolution with Integrated Positional Encoding

SR-FEINR: Continuous Remote Sensing Image Super-Resolution Using Feature-Enhanced Implicit Neural Representation

Acute mitral regurgitation after acute myocardial infarction in a patient with a patent foramen ovale: review of the diagnosis and management of acute ischemic mitral regurgitation.

Interpretable Detail-Fidelity Attention Network for Single Image Super-Resolution

A Single Image Super-Resolution Algorithm Based on Dense Residual Convolutional Network

Exponential Fusion of Interpolated Frames Network (EFIF-Net): Advancing Multi-Frame Image Super-Resolution with Convolutional Neural Networks

Arbitrary-Scale Image Generation and Upsampling using Latent Diffusion Model and Implicit Neural Decoder

An Efficient Feature Reuse Distillation Network for Lightweight Image Super-Resolution

RFCNet: Remote Sensing Image Super-Resolution Using Residual Feature Calibration Network

Dual contrastive attention-guided deformable convolutional network for single image super-resolution

Image Super-Resolution Using Very Deep Residual Channel Attention Networks

OPE-SR: Orthogonal Position Encoding for Designing a Parameter-free Upsampling Module in Arbitrary-scale Image Super-Resolution

Image super-resolution via enhanced multi-scale residual network

Activating More Information in Arbitrary-Scale Image Super-Resolution