Abstract:Spherical signals of omnidirectional videos need to be projected to a 2D plane for transmission or storage. The projection will produce geometrical deformation that affects the feature representation of Convolutional Neural Networks (CNN) on the perception of omnidirectional videos. Currently developed omnidirectional video quality assessment (OVQA) methods leverage viewport images or spherical CNN to circumvent the geometrical deformation. However, the viewport-based methods neglect the interaction between viewport images while there lacks sufficient pre-training samples for taking spherical CNN as an efficient backbone in OVQA model. In this paper, we alleviate the influence of geometrical deformation from a causal perspective. A structural causal model is adopted to analyze the implicit reason for the disturbance of geometrical deformation on quality representation and we find the latitude factor confounds the feature representation and distorted contents. Based on this evidence, we propose a Causal Intervention-based Quality prediction Network (CIQNet) to alleviate the causal effect of the confounder. The resulting framework first segments the video content into sub-areas and trains feature encoders to obtain latitude-invariant representation for removing the relationship between the latitude and feature representation. Then the features of each sub-area are aggregated by estimated weights in a backdoor adjustment module to remove the relationship between the latitude and video contents. Finally, the temporal dependencies of aggregated features are modeled to implement the quality prediction. We evaluate the performance of CIQNet on three publicly available OVQA databases. The experimental results show CIQNet achieves competitive performance against state-of-art methods. The source code of CIQNet is available at: https://github.com/Aca4peop/CIQNet.

360° video quality assessment based on saliency-guided viewport extraction

Viewport Proposal CNN for 360° Video Quality Assessment

Saliency Prediction Network for $360^\circ$ Videos

Viewport-based CNN: A Multi-task Approach for Assessing 360° Video Quality

Saliency and Depth-aware Full Reference 360-degree Image Quality Assessment

Multi-feature 360 Video Quality Estimation

Viewport-Sphere-Branch Network for Blind Quality Assessment of Stitched 360° Omnidirectional Images

Stereoscopic Video Quality Assessment Based on 3D Convolutional Neural Networks

C3DVQA: Full-Reference Video Quality Assessment with 3D Convolutional Neural Network

Omnidirectional Video Quality Assessment With Causal Intervention

No Reference Quality Assessment of Stereo Video Based on Saliency and Sparsity.

Saliency Inspired Quality Assessment of Stereoscopic 3D Video

Panoramic Video Quality Assessment Based on Non-Local Spherical CNN

Quality Assessment for Omnidirectional Video: A Spatio-Temporal Distortion Modeling Approach

Spatio-temporal salience based video quality assessment

Quality Assessment of Stereoscopic 360-degree Images from Multi-viewports

Learned Scanpaths Aid Blind Panoramic Video Quality Assessment

3D No-Reference Image Quality Assessment Via Transfer Learning and Saliency-Guided Feature Consolidation.

Optimizing Fixation Prediction Using Recurrent Neural Networks for 360$^{\circ }$ Video Streaming in Head-Mounted Virtual Reality

Multi-View Video Quality Enhancement Method Based on Multi-Scale Fusion Convolutional Neural Network and Visual Saliency

Predicting 360° Video Saliency: A ConvLSTM Encoder-Decoder Network with Spatio-temporal Consistency