Abstract:Simulating shadow interactions between real and virtual objects is important for augmented reality (AR), in which accurately and efficiently detecting real shadows from live videos is a crucial step. Most of the existing methods are capable of processing only scenes captured under a fixed viewpoint. In contrast, this article proposes a new framework for shadow detection in live outdoor videos captured under moving viewpoints. The framework splits each frame into a tracked region, which is the region tracked from the previous video frame through optical flow analysis, and an emerging region, which is newly introduced into the scene due to the moving viewpoint. The framework subsequently extracts features based on the intensity profiles surrounding the boundaries of candidate shadow regions. These features are then utilized to both correct erroneous shadow boundaries for the tracked region and to detect shadow boundaries for the emerging region by a Bayesian learning module. To remove spurious shadows, spatial layout constraints are further considered for emerging regions. The experimental results demonstrate that the proposed framework outperforms the state-of-the-art shadow tracking and detection algorithms on a variety of challenging cases in real time, including shadows on backgrounds with complex textures, nonplanar shadows, fast-moving shadows with changing typologies, and shadows cast by nonrigid objects. The quantitative experiments show that our method outperforms the best existing method, achieving a 33.3% increase in the average $F_{measure}$ on a self-collected database. Coupled with an image-based shadow-casting method, the proposed framework generates realistic shadow interaction results. This capability will be particularly beneficial for supporting AR applications.

Online Illumination Estimation of Outdoor Scenes Based on Videos Containing No Shadow Area.

Robust and Real-Time Outdoor Illumination Estimation

Computer Graphics in China: A practical approach for real-time illumination estimation of outdoor videos

On-line Illumination Estimation of Outdoor Scenes Based on Area Selection for Augmented Reality

Real-time Outdoor Illumination Estimation Based on Energy Optimization

An Algorithm for Outdoor Illumination Estimation Based on Partial Least Squares Method

A New Approach to Outdoor Illumination Estimation Based on Statistical Analysis for Augmented Reality

Lighting Simulation of Augmented Outdoor Scene Based on a Legacy Photograph.

Basis Image Decomposition for Outdoor Lighting Estimation

Basis Image Decomposition of Outdoor Time-Lapse Videos.

Light source estimation of outdoor scenes for mixed reality

Image-based Relighting from a Sparse Set of Outdoor Images.

Illumination analysis of outdoor scene images

A real-time global relighting algorithm

Shadow Estimation of Outdoor Environments Via Multi-Cue Optimization

Real-Time Shadow Detection from Live Outdoor Videos for Augmented Reality

Static Scene Illumination Estimation from Videos with Applications

Estimating Illumination Direction from Cast Shadow in Outdoor Scenes

Outdoor Shadow Estimating Using Multiclass Geometric Decomposition Based on BLS.

Sun-sky model estimation from outdoor images

Indoor Lighting Estimation using an Event Camera