Abstract:Procedural material models have been gaining traction in many applications thanks to their flexibility, compactness, and easy editability. We explore the inverse rendering problem of procedural material parameter estimation from photographs, presenting a unified view of the problem in a Bayesian framework. In addition to computing point estimates of the parameters by optimization, our framework uses a Markov Chain Monte Carlo approach to sample the space of plausible material parameters, providing a collection of plausible matches that a user can choose from, and efficiently handling both discrete and continuous model parameters. To demonstrate the effectiveness of our framework, we fit procedural models of a range of materials---wall plaster, leather, wood, anisotropic brushed metals and layered metallic paints---to both synthetic and real target images.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is: estimating the parameters of procedural material models from photographs in order to achieve physically accurate material appearance simulation. Specifically, the authors explored how to uniformly handle the inverse rendering problem of procedural material parameter estimation within the Bayesian framework. The key challenges of this problem include:
1. **Non - linear relationships**: The processes of procedural material generation and physics - based rendering are complex, resulting in non - linear and non - intuitive relationships between the procedural model parameters and the final rendering results.
2. **Loss function design**: Designing an appropriate loss function to compare the synthetic image with the target image is not obvious.
3. **Multiple solutions**: Due to the soft nature of the image - matching problem, a single best - match point estimate may be less useful than a set of reasonable matches that the user can choose.
To solve these problems, the authors proposed a computational framework based on Bayesian inference, which can handle discrete and continuous model parameters, and sample the parameter space through the Markov Chain Monte Carlo (MCMC) method, providing a series of reasonable parameter values for the user to choose. In addition, this framework also supports any differentiable BRDF models, including anisotropic and layered materials.
### Main contributions
1. **Unified view within the Bayesian framework**: Defined the posterior distribution of parameters given the captured data and prior conditions, allowing maximization and sampling of the posterior distribution.
2. **MCMC sampling method**: Introduced a Bayesian inference method that can draw samples from the reasonable material parameter space, providing more information than single - point estimates and can effectively handle discrete and continuous model parameters.
### Method overview
- **Forward evaluation process**: Procedural material generation, rendering operators, summary functions and priors jointly define the posterior distribution.
- **Loss function design**: Use neural summary functions based on VGG feature maps and hand - designed summary functions.
- **Bayesian inference**: Use MCMC methods (such as Metropolis - Hastings, Hamiltonian Monte Carlo and Metropolis - adjusted Langevin algorithms) to sample the posterior distribution.
Through these methods, the authors showed that their framework can effectively fit the procedural models of various materials, including standard isotropic dielectrics (such as plastics, leather, wall paint), dielectrics with 3D structures (such as wood), anisotropic brushed - mark metals and layered metal paints, etc.
### Example
For example, in the example of a scattering material with two parameters (scattering coefficient and phase function parameter), the posterior distribution obtained through MCMC sampling can detect the complete structure of the parameter space, which is consistent with the prediction of similarity theory. This indicates that this method can capture the similarity structure in the parameter space and help users better understand the material performance under different parameter configurations.
### Summary
This paper proposed a new computational framework that solves the key challenges in procedural material parameter estimation, especially through Bayesian inference and MCMC sampling methods, providing the user with a set of reasonable parameter values, making physically accurate material appearance simulation more flexible and efficient.