Abstract:Editing High Dynamic Range (HDR) environment maps using an inverse differentiable rendering architecture is a complex inverse problem due to the sparsity of relevant pixels and the challenges in balancing light sources and background. The pixels illuminating the objects are a small fraction of the total image, leading to noise and convergence issues when the optimization directly involves pixel values. HDR images, with pixel values beyond the typical Standard Dynamic Range (SDR), pose additional challenges. Higher learning rates corrupt the background during optimization, while lower learning rates fail to manipulate light sources. Our work introduces a novel method for editing HDR environment maps using a differentiable rendering, addressing sparsity and variance between values. Instead of introducing strong priors that extract the relevant HDR pixels and separate the light sources, or using tricks such as optimizing the HDR image in the log space, we propose to model the optimized environment map with a new variant of implicit neural representations able to handle HDR images. The neural representation is trained with adversarial perturbations over the weights to ensure smooth changes in the output when it receives gradients from the inverse rendering. In this way, we obtain novel and cheap environment maps without relying on latent spaces of expensive generative models, maintaining the original visual consistency. Experimental results demonstrate the method's effectiveness in reconstructing the desired lighting effects while preserving the fidelity of the map and reflections on objects in the scene. Our approach can pave the way to interesting tasks, such as estimating a new environment map given a rendering with novel light sources, maintaining the initial perceptual features, and enabling brush stroke-based editing of existing environment maps.

Efficient Environment Map Rendering Based on Decomposition

Efficient screen space approach for high complex dynamic scenes in real-time rendering

A Quick Raytracing Algorithm Based On The Transformation Of Environment Map

Real Time Environment Map Interpolation

Decomposition of Shading and Reflectance from a Texture Image

Predicted Virtual Soft Shadow Maps with High Quality Filtering

Accelerated Radiosity Method for Complex Environments

Accelerated Walkthroughs Of Complex Scenes Based On Visibility Culling And Image-Based Rendering

Exponential Soft Shadow Mapping

Packet-based hierarchal soft shadow mapping

Real-Time Rendering of Rough Refraction under Dynamically Varying Environmental Lighting

Rendering of Virtual Environments Based on Polygonal & Point-Based Models

Environment Maps Editing using Inverse Rendering and Adversarial Implicit Functions

Parallel and Adaptive Visibility Sampling for Rendering Dynamic Scenes with Spatially Varying Reflectance.

Multi-Resolution Rendering for Computationally Expensive Lighting Effects

Optimized Construction and Rendering of Natural Light Shaft Volumes through Depth Map

Fast, Realistic Lighting and Material Design Using Nonlinear Cut Approximation

Accelerated Walkthroughs Of Virtual Environments Based On Visibility Preprocessing And Simplification

A New Interactive Enhanced Method for Image with Poor Visibility

An Efficient Method for Rendering Detailed Soft Shadow

Parameterization-Independent Importance Sampling of Environment Maps