Abstract:Focused plenoptic cameras can record spatial and angular information of the light field (LF) simultaneously with higher spatial resolution relative to traditional plenoptic cameras, which facilitate various applications in computer vision. However, the existing plenoptic image compression methods present ineffectiveness to the captured images due to the complex micro-textures generated by the microlens relay imaging and long-distance correlations among the microimages. In this paper, a lossy end-to-end learning architecture is proposed to compress the focused plenoptic images efficiently. First, a data preprocessing scheme is designed according to the imaging principle to remove the sub-aperture image ineffective pixels in the recorded light field and align the microimages to the rectangular grid. Then, the global attention module with large receptive field is proposed to capture the global correlation among the feature maps using pixel-wise vector attention computed in the resampling process. Also, a new image dataset consisting of 1910 focused plenoptic images with content and depth diversity is built to benefit training and testing. Extensive experimental evaluations demonstrate the effectiveness of the proposed approach. It outperforms intra coding of HEVC and VVC by an average of 62.57% and 51.67% bitrate reduction on the 20 preprocessed focused plenoptic images, respectively. Also, it achieves 18.73% bitrate saving and generates perceptually pleasant reconstructions compared to the state-of-the-art end-to-end image compression methods, which benefits the applications of focused plenoptic cameras greatly. The dataset and code are publicly available at https://github.com/VincentChandelier/GACN.

Pixel Gradient Based Zooming Method for Plenoptic Intra Prediction

Zoomable intra prediction for multi-focus plenoptic 2.0 video coding

Plenoptic 2.0 Intra Coding Using Imaging Principle

Feature Based Inter Prediction Optimization for Non-Translational Video Coding in Cloud

Imaging-Correlated Intra Prediction for Plenoptic 2.0 Video Coding

Research on Zoom Algorithm Based on Dual Resolution Camera

Plenoptic Image Coding Using Macropixel-Based Intra Prediction

Macropixel Based Fast Motion Estimation for Plenoptic Video Compression.

Macropixel-constrained Collocated Position Search for Plenoptic Video Coding

Spatial Information Refinement for Chroma Intra Prediction in Video Coding

Lenslet Image Compression Using Adaptive Macropixel Prediction

Plenoptic Image Compression via Simplified Subaperture Projection.

Lenslet Image Compression Based on Image Reshaping and Macro-Pixel Intra Prediction

Microimage-based Two-step Search For Plenoptic 2.0 Video Coding

Image Reshaping for Efficient Compression of Plenoptic Content

Learned Focused Plenoptic Image Compression with Microimage Preprocessing and Global Attention

Gradient Based Interpolation For Intra Angular Prediction In Hevc

Chroma Intra Prediction with Lightweight Attention-Based Neural Networks

Panoramic Video Inter Frame Prediction and Viewport Prediction Based on Background Modeling

Plenoptic Point Cloud Compression Using Multiview Extension of High Efficiency Video Coding