EAT: epipolar-aware Transformer for low-light light field enhancement

Xingzheng Wang,Wenhao Huang,Kaiqiang Chen,Zixuan Wang,Yuanlong Deng
DOI: https://doi.org/10.1007/s11042-024-19271-z
IF: 2.577
2024-05-18
Multimedia Tools and Applications
Abstract:Current low-light light field (LF) enhancement methods are mainly based on the convolutional neural network and have achieved considerable effect. However, these methods only consider neighboring views or local epipolar plane images (EPIs), which neglect the rich epipolar geometry among all views and fail to restore more texture details. In this paper, we deliberate the epipolar geometry and take effective utilization of epipolar geometry as a principle to design an epipolar-aware Transformer (EAT). On the one hand, we develop a global epipolar geometry (GEG) structure by processing all views in different epipolar directions respectively to efficiently extract epipolar geometry. On the other hand, in order to exploit epipolar geometry further, we propose an attention-based view selection strategy employing the Transformer to generate attention maps across views and focus on more significant views in the corresponding directions. Specially, EAT consists of two stages: in Stage-I, all views are processed based on GEG structure and selectively fused into the global representation. And in Stage-II, the global information is fine-tuned to self-adaptively enhance each view based on their different angular positions. Experimental results on three publicly available LF datasets show that the proposed method outdoes the state-of-the-art methods by 0.797 dB in average PSNR gain.
computer science, information systems, theory & methods,engineering, electrical & electronic, software engineering
What problem does this paper attempt to address?