Abstract:Compared with the annotated data for the 2D image saliency prediction task, the annotated data for training om-nidirectional image (or 360 degrees image) saliency prediction models are not sufficient. Most existing fully-supervised saliency prediction methods for omnidirectional images (ODIs) adopt a scheme, first training the methods on a labeled large 2D image saliency prediction dataset and then fine-tuning the methods on the labeled tiny ODI sa-liency prediction dataset. However, this strategy is time-consuming and may not inadequately mine the visual features built in ODIs. To explore the visual attributes targeted at ODIs and address the shortage of labels on these ODIs, in this paper, we propose an end-to-end semi-supervised network, namely VFNet, which relies on viewport features and only utilizes ODIs as training data, for ODI saliency prediction. Concretely, we adopt con-sistency regularization as our semi-supervised learning framework. The predictions between main and auxiliary saliency inference networks in the VFNet enforce consistency. Aiming at ODIs, we introduce a new form of per-turbation, i.e., DropView, to improve the effectiveness of consistency regularization. By randomly dropping out different 360 degrees cubemap viewport features before the auxiliary saliency inference network, the proposed DropView enhances the robustness of the final ODI saliency prediction. To adaptively interact with the equirectangular and different cubemap viewport features according to their contributions, we introduce a Viewport Feature Adaptive Integration (VFAI) module and deploy the VFAI module at different levels in the VFNet to raise the capacity of feature encoding of our VFNet. Compared with state-of-the-art fully-supervised methods, our VFNet with fewer labeled training data achieves competitive performance demonstrated by exten-sive experiments on two publicly available datasets. (c) 2022 Elsevier B.V. All rights reserved.

Omnidirectional Image Super-Resolution Via Latitude Adaptive Network.

Omnidirectional image super-resolution via position attention network

Orientation-Aware Deep Neural Network for Real Image Super-Resolution.

DuDoINet: Dual-Domain Implicit Network for Multi-Modality MR Image Arbitrary-scale Super-Resolution

Deep Networks with Detail Enhancement for Infrared Image Super-Resolution

OmniSSR: Zero-shot Omnidirectional Image Super-Resolution using Stable Diffusion Model

OPDN: Omnidirectional Position-aware Deformable Network for Omnidirectional Image Super-Resolution

Single Remote Sensing Image Super-Resolution Via a Generative Adversarial Network with Stratified Dense Sampling and Chain Training

Spatio-Temporal Distortion Aware Omnidirectional Video Super-Resolution

Super-resolution of Omnidirectional Images Using Adversarial Learning

Non-Locally up-Down Convolutional Attention Network for Remote Sensing Image Super-Resolution

Sparse-Based Domain Adaptation Network for OCTA Image Super-Resolution Reconstruction

Exploring Viewport Features for Semi-Supervised Saliency Prediction in Omnidirectional Images

Omni Aggregation Networks for Lightweight Image Super-Resolution

OPE-SR: Orthogonal Position Encoding for Designing a Parameter-free Upsampling Module in Arbitrary-scale Image Super-Resolution

OSRT: Omnidirectional Image Super-Resolution with Distortion-aware Transformer

Non-Local Nested Residual Attention Network for Stereo Image Super-Resolution.

A Closed-Loop Network for Single Infrared Remote Sensing Image Super-Resolution in Real World

Closed-Loop Matters: Dual Regression Networks for Single Image Super-Resolution

DASRSNet: Multitask Domain Adaptation for Super-Resolution-Aided Semantic Segmentation of Remote Sensing Images

A Super-Resolution Algorithm Based on Hybrid Network for Multi-Channel Remote Sensing Images