Abstract:Due to recent advancements in virtual reality (VR) and augmented reality (AR), the demand for high quality immersive contents is a primary concern for production companies and consumers. Similarly, the topical record-breaking performance of deep learning in various domains of artificial intelligence has extended the attention of researchers to contribute to different fields of computer vision. To ensure the quality of immersive media contents using these advanced deep learning technologies, several learning based Stitched Image Quality Assessment methods have been proposed with reasonable performances. However, these methods are unable to localize, segment, and extract the stitching errors in panoramic images. Further, these methods used computationally complex procedures for quality assessment of panoramic images. With these motivations, in this paper, we propose a novel three-fold Deep Learning based No-Reference Stitched Image Quality Assessment (DLNR-SIQA) approach to evaluate the quality of immersive contents. In the first fold, we fined-tuned the state-of-the-art Mask R-CNN (Regional Convolutional Neural Network) on manually annotated various stitching error-based cropped images from the two publicly available datasets. In the second fold, we segment and localize various stitching errors present in the immersive contents. Finally, based on the distorted regions present in the immersive contents, we measured the overall quality of the stitched images. Unlike existing methods that only measure the quality of the images using deep features, our proposed method can efficiently segment and localize stitching errors and estimate the image quality by investigating segmented regions. We also carried out extensive qualitative and quantitative comparison with full reference image quality assessment (FR-IQA) and no reference image quality assessment (NR-IQA) on two publicly available datasets, where the proposed system outperformed the existing state-of-the-art techniques.

Spatio-Temporal Feature Integration for Quality Assessment of Stitched Omnidirectional Images

Attentive Deep Image Quality Assessment for Omnidirectional Stitching

Viewport-Sphere-Branch Network for Blind Quality Assessment of Stitched 360° Omnidirectional Images

Attentive Deep Stitching and Quality Assessment for 360 $^{\circ }$ Omnidirectional Images

Image Quality Assessment for Omnidirectional Cross-reference Stitching

Cross-Reference Stitching Quality Assessment for 360° Omnidirectional Images

VSOIQE: A Novel Viewport-Based Stitched 360° Omnidirectional Image Quality Evaluator

Towards All Weather and Unobstructed Multi-Spectral Image Stitching: Algorithm and Benchmark

Omnidirectional Image Quality Assessment by Distortion Discrimination Assisted Multi-Stream Network

Learning Spatiotemporal Interactions for User-Generated Video Quality Assessment

Monocular and Binocular Interactions Oriented Deformable Convolutional Networks for Blind Quality Assessment of Stereoscopic Omnidirectional Images

Dual-stream Perception-driven Blind Quality Assessment for Stereoscopic Omnidirectional Images

DLNR-SIQA: Deep Learning-Based No-Reference Stitched Image Quality Assessment

An Error-Activation-Guided Blind Metric for Stitched Panoramic Image Quality Assessment

Quality Assessment for Omnidirectional Video: A Spatio-Temporal Distortion Modeling Approach

Multi-Spectral Image Stitching via Spatial Graph Reasoning

Quality Assessment for Stitched Panoramic Images via Patch Registration and Bidimensional Feature Aggregation

Content-Seam-Preserving Multi-Alignment Network for Visual-Sensor-Based Image Stitching

OVQE: Omniscient Network for Compressed Video Quality Enhancement

Omnidirectional Video Quality Assessment With Causal Intervention

Stereoscopic Omnidirectional Image Quality Assessment Based on Predictive Coding Theory