Abstract:Global context information is particularly important for comprehensive scene understanding. It helps clarify local confusions and smooth predictions to achieve fine-grained and coherent results. However, most existing light field processing methods leverage convolution layers to model spatial and angular information. The limited receptive field restricts them to learn long-range dependency in LF structure. In this article, we propose a novel network based on deep efficient transformers ( i.e., LF-DET) for LF spatial super-resolution. It develops a spatial-angular separable transformer encoder with two modeling strategies termed as sub-sampling spatial modeling and multi-scale angular modeling for global context interaction. Specifically, the former utilizes a sub-sampling convolution layer to alleviate the problem of huge computational cost when capturing spatial information within each sub-aperture image. In this way, our model can cascade more transformers to continuously enhance feature representation with limited resources. The latter processes multi-scale macro-pixel regions to extract and aggregate angular features focusing on different disparity ranges to well adapt to disparity variations. Besides, we capture strong similarities among surrounding pixels by dynamic positional encodings to fill the gap of transformers that lack of local information interaction. The experimental results on both real-world and synthetic LF datasets confirm our LF-DET achieves a significant performance improvement compared with state-of-the-art methods. Furthermore, our LF-DET shows high robustness to disparity variations through the proposed multi-scale angular modeling.

Efficient Light Field Image Super-Resolution Via Progressive Disentangling

Disentangling Light Fields for Super-Resolution and Disparity Estimation

Light Field Image Super-Resolution Network Based on Angular Difference Enhancement

Light Field Image Super-Resolution Using Deformable Convolution

Light field angular super-resolution based on structure and scene information

DistgEPIT: Enhanced Disparity Learning for Light Field Image Super-Resolution

Did: Dimension-Invariant Disentangling Model for Light Field Spatial Super-Resolution

Light Field Image Super-Resolution with Transformers

Exploiting Spatial and Angular Correlations with Deep Efficient Transformers for Light Field Image Super-Resolution

Light Field Angular Super-Resolution Network Based on Convolutional Transformer and Deep Deblurring

Angular-flexible network for light field image super-resolution

Light Field Image Super-Resolution Based on Raw Data with Transformers

Spatial-Angular Interaction for Light Field Image Super-Resolution

Light Field Image Super-Resolution Based on Dual Learning and Deep Fourier Channel Attention

Dense Dual-Attention Network for Light Field Image Super-Resolution

A Coarse-to-Fine Convolutional Neural Network for Light Field Angular Super-Resolution

Progressive Multi-Scale Fusion Network for Light Field Super-Resolution

Light Field Super-Resolution Based on Spatial and Angular Attention

Disentangled Feature Distillation for Light Field Super-Resolution with Degradations

Light Field Reconstruction using Efficient Pseudo 4D Epipolar-Aware Structure

Light Field Angular Super-Resolution Via Dense Correspondence Field Reconstruction