Abstract:For convenient transmission, omnidirectional images (ODIs) usually follow the equirectangular projection (ERP) format and are low-resolution. To provide better immersive experience, omnidirectional image super resolution (ODISR) is essential. However, ERP ODIs suffer from serious geometric distortion and pixel stretching across latitudes, generating massive redundant information at high latitudes. This characteristic poses a huge challenge for the traditional SR methods, which can only obtain the suboptimal ODISR performance. To address this issue, we propose a novel position attention network (PAN) for ODISR in this paper. Specifically, a two-branch structure is introduced, in which the basic enhancement branch (BE) serves to achieve coarse deep feature enhancement for extracted shallow features. Meanwhile, the position attention enhancement branch (PAE) builds a positional attention mechanism to dynamically adjust the contribution of features at different latitudes in the ERP representation according to their positions and stretching degrees, which achieves the enhancement for the differentiated information, suppresses the redundant information, and modulate the deep features with spatial distortion. Subsequently, the features of two branches are fused effectively to achieve the further refinement and adapt the distortion characteristic of ODIs. After that, we exploit a long-term memory module (LM), promoting information interactions and fusions between the branches to enhance the perception of the distortion, aggregating the prior hierarchical features to keep the long-term memory and boosting the ODISR performance. Extensive results demonstrate the state-of-the-art performance and the high efficiency of our PAN in ODISR.

OIDC-Net: Omnidirectional Image Distortion Correction via Coarse-to-Fine Region Attention

Omnisupervised Omnidirectional Semantic Segmentation

Stereo Calibration and Rectification for Omnidirectional Multi-Camera Systems

Distortion-Tolerant Monocular Depth Estimation On Omnidirectional Images Using Dual-cubemap

Geometric Distortion Guided Transformer for Omnidirectional Image Super-Resolution

A Deep Ordinal Distortion Estimation Approach for Distortion Rectification

Omnidirectional Image Quality Assessment by Distortion Discrimination Assisted Multi-Stream Network

OmniFusion: 360 Monocular Depth Estimation via Geometry-Aware Fusion

RecRecNet: Rectangling Rectified Wide-Angle Images by Thin-Plate Spline Model and DoF-based Curriculum Learning

Look at the Neighbor: Distortion-aware Unsupervised Domain Adaptation for Panoramic Semantic Segmentation

Orientation-Aware Deep Neural Network for Real Image Super-Resolution.

Omnidirectional image super-resolution via position attention network

Attentive Deep Image Quality Assessment for Omnidirectional Stitching

OSRT: Omnidirectional Image Super-Resolution with Distortion-aware Transformer

An efficient deep learning-based framework for image distortion correction

Multi-Projection Fusion and Refinement Network for Salient Object Detection in 360° Omnidirectional Image

Distortion-aware Transformer in 360° Salient Object Detection

OmiQnet: Multiscale feature aggregation convolutional neural network for omnidirectional image assessment

SelfOdom: Self-supervised Egomotion and Depth Learning via Bi-directional Coarse-to-Fine Scale Recovery

RoFIR: Robust Fisheye Image Rectification Framework Impervious to Optical Center Deviation

DaFIR: Distortion-aware Representation Learning for Fisheye Image Rectification