Intra-Inter View Interaction Network for Light Field Image Super-Resolution

Gaosheng Liu,Huanjing Yue,Jiamin Wu,Jingyu Yang

DOI: https://doi.org/10.1109/tmm.2021.3124385

IF: 7.3

2023-01-01

IEEE Transactions on Multimedia

Abstract:Light field (LF) cameras, which can record real-word scenes from multiple viewpoints in a single shot, are widely used in 3D reconstruction, re-focusing, and virtual reality etc . However, the inherent trade-off between spatial resolution and angular resolution of LF images hinders their applications for scenarios requiring high resolutions. In this paper, we propose a novel intra-inter view interaction network for LF image super-resolution, termed as LF-IINet, to exploit the correlations among all views and simultaneously preserve the parallax structure of LF views. The proposed LF-IINet consists of two parallel branches. Specifically, the top branch extracts global inter-view information, and the bottom branch first independently maps each view to deep representations and then models the correlations among all intra-view features via proposed multi-view context block (MCB). The two branches interact with each other by proposed inter-assist-intra feature updating module (IntraFUM, where the intra feature are updated with the assistance of the inter feature) and intra-assist-inter feature updating module (InterFUM, where the inter feature are updated with the assistance of the intra feature). In this way, our LF-IINet incorporates rich angular and spatial information for LF image super-resolution. Extensive comparison with state-of-the-art methods demonstrates that our method achieves superior performance visually and quantitatively. Furthermore, quantitative results also show that our method is effective for LF images with either small or large disparities. Our code is shared in https://github.com/GaoshengLiu/LF-IINet .