Assessing Photorealism of Rendered Objects in Real-World Images: A Transparent and Reproducible User Study

Sven Kluge,Oliver Staadt

2024-07-02

Abstract:In an era where numerous studies claim to achieve almost photorealism with real-time automated environment capture, there is a need for assessments and reproducibility in this domain. This paper presents a transparent and reproducible user study aimed at evaluating the photorealism of real-world images composed with virtual rendered objects, that have been generated using classical environment capturing and rendering techniques. We adopted a two-alternative forced choice methodology to compare pairs of images created by integrating virtual objects into real photographs, following a classic pipeline. A control group with defined directional light parameters was included to validate the study's correctness. The findings revealed some insights, suggesting that observers experienced difficulties in differentiating between rendered and real objects. This work establishes the groundwork for future studies, aimed at enhancing the visual fidelity and realism of virtual objects in real-world environments.

Human-Computer Interaction,Graphics

What problem does this paper attempt to address?

The aim of this paper is to address the problem of evaluating the realism of rendered objects in real images. Specifically, the researchers conducted a transparent and repeatable user study to assess the realism of combining real-world images generated using classic environment capture and rendering techniques with virtual rendered objects. The study employed a two-alternative forced choice method, comparing image pairs created by integrating virtual objects into real photos, and included a control group with defined directional light source parameters to validate the study's effectiveness.

The results indicate that observers have some difficulty distinguishing between rendered objects and real objects, laying the groundwork for further improving the visual fidelity and realism of virtual objects in real environments. Additionally, the study found that shadows of different hardness significantly impact perceived realism, especially in the case of directional light source rendering (e.g., dir1, dir5, dir10), with dir1 (1-degree shadow penumbra angle) having the lowest realism. However, there were no significant differences between images rendered with dir5, dir10, and HDRI environment maps, suggesting that the realism of images in these groups is similar and difficult to distinguish.

In conclusion, the main contribution of this paper is providing a repeatable research method for evaluating the realism of virtual objects in real images, and offering benchmarks and references for future research.