Light field angular super-resolution based on structure and scene information

Jiangxin Yang,Lingyu Wang,Lifei Ren,Yanpeng Cao,Yanlong Cao
DOI: https://doi.org/10.1007/s10489-022-03759-y
IF: 5.3
2022-01-01
Applied Intelligence
Abstract:Light fields contain a wealth of information about real-world scenes and can be easily acquired by commercial light field cameras. However, the trade-off between angular and spatial resolution is inevitable. This paper proposes an end-to-end light field angular super-resolution network by exploiting structure and scene information to mitigate this problem. First, a Light Field ResBlock is proposed to explicitly extract epipolar plane image features and scene features. The epipolar plane image features with obvious directionality represent light field structure information that contains correlations among different views, such as the intensity and depth consistencies. The scene features express scene information that depicts local details and global information and describes individual objects in a scene. The exploitation of light field structure and scene information can gather the intrinsic characteristics of light fields. Then, 4D deconvolution is employed to upsample the angular resolution based on the extracted features related to light field structure and scene. The utilization of 4D deconvolution preserves the connections among sparse views and generates new views with high correlations. Last but not least, the refinement network further exploiting LF structure and scene information is employed to alleviate the artifacts induced by too little angular information during the early stage of the model. Experiments on two different angular super-resolution tasks demonstrate that our method can achieve the best angular resolution enhancement performance on different datasets.
What problem does this paper attempt to address?