Panoramic Light Field from Hand-Held Video and Its Sampling for Real-Time Rendering

Qiang Zhao,Feng Dai,Jing Lv,Yike Ma,Yongdong Zhang

DOI: https://doi.org/10.1109/tcsvt.2019.2900051

IF: 5.859

2019-01-01

IEEE Transactions on Circuits and Systems for Video Technology

Abstract:By providing angular and spatial information of light rays, light field images are widely used in many applications. To capture large field of view light field, existing approaches either stitch small field of view light fields, whose apertures are also small, or leverage specialized equipments, which are not accessible to ordinary users. In this paper, we present a method to extract fully 360° field of view panoramic light fields from densely sampled hand-held videos. Unlike previous works, our method handles the large exposure variation and motion blur problems, which are common in the panoramic capture and hand-held videos. To provide real-time rendering performance, our method applies light field sampling to the extracted panoramic light field. Based on the unstructured representation, our method can sample the light field without explicit parameterization. We formulate the sampling problem into a set multicover problem and solve it globally using integer linear programming. Compared with the greedy based method, our approach can use much less video frames to represent the panoramic light field, but still can get better rendering results.