Spatial-angular interaction for arbitrary scale light field reconstruction
Sen Xiang,Weijie Chen,Jin Wu
DOI: https://doi.org/10.1007/s11042-024-18714-x
IF: 2.577
2024-03-12
Multimedia Tools and Applications
Abstract:Light field records both the intensity and the direction of light, and thus facilitates a range of applications, but the huge amount of data poses great challenges to storage and transmission. To address this issue, numerous methods have been developed to reconstruct dense light fields from sparse ones. However, the existing approaches are limited to fixed angular upscaling factors. In this paper, we propose an end-to-end meta-learning-based spatial-angular interaction approach to generate dense light-field images at arbitrary positions. Unlike conventional methods, our model uses meta-learning to predict the weights in view synthesis, enabling the generation of dense light field images at arbitrary viewpoints and scales. Furthermore, to extract both spatial and angular features more precisely, we utilize the macro-pixel convolution which can extract three types of information: spatial, horizontal angular, and vertical angular ones. Experimental results demonstrate that the proposed model can generate novel viewpoints at any position, and reconstruct light fields with any up-scaling factors. The reconstructed light fields are of high quality with 4.2dB PSNR improvement and 0.038 SSIM gain over the second-best method.
computer science, information systems, theory & methods,engineering, electrical & electronic, software engineering