Projection Mapping under Environmental Lighting by Replacing Room Lights with Heterogeneous Projectors

Masaki Takeuchi,Hiroki Kusuyama,Daisuke Iwai,Kosuke Sato

2024-03-05

Abstract:Projection mapping (PM) is a technique that enhances the appearance of real-world surfaces using projected images, enabling multiple people to view augmentations simultaneously, thereby facilitating communication and collaboration. However, PM typically requires a dark environment to achieve high-quality projections, limiting its practicality. In this paper, we overcome this limitation by replacing conventional room lighting with heterogeneous projectors. These projectors replicate environmental lighting by selectively illuminating the scene, excluding the projection target. Our contributions include a distributed projector optimization framework designed to effectively replicate environmental lighting and the incorporation of a large-aperture projector, in addition to standard projectors, to reduce high-luminance emitted rays and hard shadows -- undesirable factors for collaborative tasks in PM. We conducted a series of quantitative and qualitative experiments, including user studies, to validate our approach. Our findings demonstrate that our projector-based lighting system significantly enhances the contrast and realism of PM results even under environmental lighting compared to typical lights. Furthermore, our method facilitates a substantial shift in the perceived color mode from the undesirable aperture-color mode, where observers perceive the projected object as self-luminous, to the surface-color mode in PM.

Graphics,Human-Computer Interaction

What problem does this paper attempt to address?

This paper mainly addresses the problem of quality degradation of Projection Mapping (PM) under ambient light. Traditional PM techniques require a dark environment to ensure high-quality projection, but this limits its practical application. The paper presents a new method that uses heterogeneous projectors to replace conventional room lighting fixtures to simulate ambient light while improving projection contrast.

Specifically, the contributions of the paper include:
1. Proposing a distributed projector optimization framework that effectively simulates ambient lighting, and utilizes large-aperture projectors to reduce high-intensity light and hard shadows, which are undesirable in collaborative tasks.
2. Through a series of quantitative and qualitative experiments, including user studies, validating that this method significantly enhances the contrast and realism of PM results under ambient light, while transforming the perceived color mode from the non-ideal "light source color mode" to the "surface color mode".

The research also points out that existing PM techniques make projected objects appear self-illuminated in dark environments, whereas their method makes the projection results closer to realistically illuminated objects under ambient light. The paper demonstrates the effectiveness and feasibility of the proposed method through prototype installation and experiments, and confirms its improvement in the visual effects of PM under ambient light through user studies.