Abstract:Stereoscopic projection mapping (PM) allows a user to see a three-dimensional (3D) computer-generated (CG) object floating over physical surfaces of arbitrary shapes around us using projected imagery. However, the current stereoscopic PM technology only satisfies binocular cues and is not capable of providing correct focus cues, which causes a vergence--accommodation conflict (VAC). Therefore, we propose a multifocal approach to mitigate VAC in stereoscopic PM. Our primary technical contribution is to attach electrically focus-tunable lenses (ETLs) to active shutter glasses to control both vergence and accommodation. Specifically, we apply fast and periodical focal sweeps to the ETLs, which causes the "virtual image'" (as an optical term) of a scene observed through the ETLs to move back and forth during each sweep period. A 3D CG object is projected from a synchronized high-speed projector only when the virtual image of the projected imagery is located at a desired distance. This provides an observer with the correct focus cues required. In this study, we solve three technical issues that are unique to stereoscopic PM: (1) The 3D CG object is displayed on non-planar and even moving surfaces; (2) the physical surfaces need to be shown without the focus modulation; (3) the shutter glasses additionally need to be synchronized with the ETLs and the projector. We also develop a novel compensation technique to deal with the "lens breathing" artifact that varies the retinal size of the virtual image through focal length modulation. Further, using a proof-of-concept prototype, we demonstrate that our technique can present the virtual image of a target 3D CG object at the correct depth. Finally, we validate the advantage provided by our technique by comparing it with conventional stereoscopic PM using a user study on a depth-matching task.

Animated Stickies: Fast Video Projection Mapping onto a Markerless Plane through a Direct Closed-Loop Alignment

Dynamic Projection Mapping onto Deforming Non-Rigid Surface Using Deformable Dot Cluster Marker

Reflex Spatial Light Modulator Based on Micromirror Array Device

Device and method for 360-degree full-parallax three dimensional display based on high-speed projector

Multi-projector Calibration and Alignment Using Flatness Analysis for Irregular-Shape Surfaces

FibAR: Embedding Optical Fibers in 3D Printed Objects for Active Markers in Dynamic Projection Mapping

Dynamic depth-of-field projection mapping method based on a variable focus lens and visual feedback

Fabricating Diminishable Visual Markers for Geometric Registration in Projection Mapping

A Penetrable Interactive 3D Display Based on Motion Recognition （invited Paper）

A Real-Time Interactive System of Surface Reconstruction and Dynamic Projection Mapping with RGB-depth Sensor and Projector.

P‐78: an Interactive 360° Floating 3D Display Based on Gesture Recognition

Performance Capture of High-Speed Motion Using Staggered Multi-View Recording

A projection mapping method of a video in a three-dimensional scene based on a screen space

An Intelligent Projection System Adapted to Arbitrary Surfaces

A Two-Dimensional Location Method Based On Digital Micro-Mirror Device Used In Interactive Projection Systems

Multifocal Stereoscopic Projection Mapping

Building Virtual Entertainment Environment with Tiled Display Wall and Motion Tracking.

Motion-Blur-Free High-Speed Video Shooting Using a Resonant Mirror

Multi-Projector Tiled Display Wall Calibration With A Camera

High-resolution Volumetric Display System with Multi-screen in Parallel Motion

FlyCap: Markerless Motion Capture Using Multiple Autonomous Flying Cameras