Abstract:Single image scene relighting aims to generate a realistic new version of an input image so that it appears to be illuminated by a new target light condition. Although existing works have explored this problem from various perspectives, generating relit images under arbitrary light conditions remains highly challenging, and related datasets are scarce. Our work addresses this problem from both the dataset and methodological perspectives. We propose two new datasets: a synthetic dataset with the ground truth of intrinsic components and a real dataset collected under laboratory conditions. These datasets alleviate the scarcity of existing datasets. To incorporate physical consistency in the relighting pipeline, we establish a two-stage network based on intrinsic decomposition, giving outputs at intermediate steps, thereby introducing physical constraints. When the training set lacks ground truth for intrinsic decomposition, we introduce an unsupervised module to ensure that the intrinsic outputs are satisfactory. Our method outperforms the state-of-the-art methods in performance, as tested on both existing datasets and our newly developed datasets. Furthermore, pretraining our method or other prior methods using our synthetic dataset can enhance their performance on other datasets. Since our method can accommodate any light conditions, it is capable of producing animated results. The dataset, method, and videos are publicly available.

Interactive Image Based Relighting With Physical Light Acquisition

An Approximate Shading Model with Detail Decomposition for Object Relighting

Relighting with real incident light source

An Approximate Shading Model For Object Relighting

Image-based Relighting from a Sparse Set of Outdoor Images.

A real-time global relighting algorithm

Interactive Relighting of Dynamic Refractive Objects

Interactive Relighting with Dynamic BRDFs

Physically-Based Editing of Indoor Scene Lighting from a Single Image

Relighting Abstracted Image Via Salient Edge-Guided Luminance Field Optimization.

Relighting in spatial augmented reality

Designing An Illumination-Aware Network for Deep Image Relighting

3D object relighting based on multi-view stereo and image based lighting techniques

Estimating Illumination Parameters in Real Space with Application to Image Relighting

Single View Relighting by Normal Map Painting

Weakly-supervised Single-view Image Relighting

Relighting from a Single Image: Datasets and Deep Intrinsic-based Architecture

Single Image Based Illumination Estimation for Lighting Virtual Object in Real Scene

Illumination brush: interactive design of image-based lighting.

Single-view relighting with normal map painting

PNRNet: Physically-Inspired Neural Rendering for Any-to-Any Relighting.