Abstract:For deep learning methods of image superresolution, the most critical issue is whether the paired low and high resolution images for training accurately reflect the sampling process of real cameras. Low and high resolution (LR∼HR) image pairs synthesized by existing degradation models (e.g., bicubic downsampling) deviate from those in reality; thus the superresolution CNN trained by these synthesized LR∼HR image pairs does not perform well when being applied to real images. In this paper, we propose a novel method to capture a large set of realistic LR∼HR image pairs using real cameras. The data acquisition is carried out under controllable lab conditions with minimum human intervention and at high throughput (about 500 image pairs per hour). The high level of automation makes it easy to produce a set of real LR∼HR training image pairs for each camera. Our innovation is to shoot images displayed on an ultra-high quality screen at different resolutions. There are three distinctive advantages of our method for image superresolution. First, as the LR and HR images are taken of a 3D planar surface (the screen) the registration problem fits exactly to a homography model and we can display specially designed markers on the image to improve the registration precision. Second, the displayed digital image file can be exploited as a reference to optimize the high frequency content of the restored image. Third, this high-efficiency data collection method makes it possible to collect a customized dataset for each camera sensor, for which one can train a specific model for the intended camera sensor. Experimental results show that training a superresolution CNN by our LR∼HR dataset has superior restoration performance than training it by existing datasets on real world images at the inference stage.

Synthesizing cloth wrinkles by CNN-based geometry image superresolution: Synthesizing cloth wrinkles by CNN-based geometry image SR

Multi-Feature Super-Resolution Network for Cloth Wrinkle Synthesis

Neural Garment Dynamic Super-Resolution

SR-USRN: Learning Image Super-Resolution with Unified Structure and Reverse Network.

SelFSR: Self-Conditioned Face Super-Resolution in the Wild via Flow Field Degradation Network

SelFSR: Self-Conditioned Face Super-Resolution in the Wild Via Flow Field Degradation Network

DeepWrinkles: Accurate and Realistic Clothing Modeling

An Efficient Dual-reference Training Data Acquisition Method for CNN Image Super-Resolution

Realtime Simulation of Thin-Shell Deformable Materials using CNN-Based Mesh Embedding

Spatially Adaptive Cloth Regression with Implicit Neural Representations

Geometry-Aware Reference Synthesis for Multi-View Image Super-Resolution.

Face Super-Resolution Via Multilayer Locality-Constrained Iterative Neighbor Embedding and Intermediate Dictionary Learning.

Learning Cross-scale Correspondence and Patch-based Synthesis for Reference-based Super-Resolution.

Face Super-Resolution Via Hierarchical Multi-Scale Residual Fusion Network.

Unsupervised Real-World Image Super-Resolution via Dual Synthetic-to-Realistic and Realistic-to-Synthetic Translations

Surface‐aware Mesh Texture Synthesis with Pre‐trained 2D CNNs

Data Acquisition and Preparation for Dual-reference Deep Learning of Image Super-Resolution

CrossNet: An End-to-End Reference-Based Super Resolution Network Using Cross-Scale Warping

3D Clothing Wrinkle Modeling Method Based on Gaussian Curvature and Deep Learning

Spatial-Temporal Recurrent Residual Networks for Video Super-Resolution.

Video Super-Resolution Based on Spatial-Temporal Recurrent Residual Networks