Abstract:High-quality panoramic images with a Field of View (FoV) of 360° are essential for contemporary panoramic computer vision tasks. However, conventional imaging systems come with sophisticated lens designs and heavy optical components. This disqualifies their usage in many mobile and wearable applications where thin and portable, minimalist imaging systems are desired. In this paper, we propose a Panoramic Computational Imaging Engine (PCIE) to achieve minimalist and high-quality panoramic imaging. With less than three spherical lenses, a Minimalist Panoramic Imaging Prototype (MPIP) is constructed based on the design of the Panoramic Annular Lens (PAL), but with low-quality imaging results due to aberrations and small image plane size. We propose two pipelines, i.e. Aberration Correction (AC) and Super-Resolution and Aberration Correction (SR&AC), to solve the image quality problems of MPIP, with imaging sensors of small and large pixel size, respectively. To leverage the prior information of the optical system, we propose a Point Spread Function (PSF) representation method to produce a PSF map as an additional modality. A PSF-aware Aberration-image Recovery Transformer (PART) is designed as a universal network for the two pipelines, in which the self-attention calculation and feature extraction are guided by the PSF map. We train PART on synthetic image pairs from simulation and put forward the PALHQ dataset to fill the gap of real-world high-quality PAL images for low-level vision. A comprehensive variety of experiments on synthetic and real-world benchmarks demonstrates the impressive imaging results of PCIE and the effectiveness of the PSF representation. We further deliver heuristic experimental findings for minimalist and high-quality panoramic imaging. Our dataset and code will be available at <a class="link-external link-https" href="https://github.com/zju-jiangqi/PCIE-PART" rel="external noopener nofollow">this https URL</a>.

A Panorama-Based Architecture For Multimedia From Hand-Held Devices

Web-based virtual walkthrough of panoramas

Universal Browse and Display Framework for Ultra-Resolution Image

PanoWalk: a remote image-based rendering system for mobile devices

Real-time viewing of large images based on multi-core

Interactive Browsing of Large Images on Multi-Projector Display Wall System

Mpano: Cloud-Based Mobile Panorama View from Single Picture

IMAGE-BASED WALKTHROUGH OVER THE INTERNET

Image-Based Walkthrough over Internet on Mobile Devices.

Panoramic Video Streaming Based on Splitting

Multi-Viewpoint Panorama Construction with Wide-Baseline Images

Fast Generation Of Dynamic And Multi-Resolution 360 Degrees Panorama From Video Sequences

Virtual Walk-Through of Dunhuang Panorama-Based on Web

Transcoing-based Data Transmission of Sphere Panoramic Videos

Title Data compression and transmission aspects of panoramicvideos

Fast Construction of Dynamic and Multi-Resolution 360° Panoramas from Video Sequences

A robust image mosaicing technique capable of creating integrated panoramas

PanoSwarm: Collaborative and Synchronized Multi-Device Panoramic Photography

A Feature-Based Approach to Panorama Generation

A Multi-Camera Approach to Image-Based Rendering and 3-D/multiview Display of Ancient Chinese Artifacts

Minimalist and High-Quality Panoramic Imaging with PSF-aware Transformers