Abstract:The depth-image-based rendering (DIBR) algorithms used for 3D video applications introduce new types of artifacts mostly located around the disoccluded regions. As the DIBR algorithms involve geometric transformations, most of them introduce non-uniform geometric distortions affecting the edge coherency in the synthesized images. Such distortions are not handled efficiently by the common image quality assessment metrics which are primarily designed for other types of distortions. In order to better deal with specific geometric distortions in the DIBR-synthesized images, we propose a full-reference metric based on multi-scale image decomposition applying morphological filters. Using non-linear morphological filters in multi-scale image decomposition, important geometric information such as edges is maintained across different resolution levels. Edge distortion between the multi-scale representation subbands of the reference image and the DIBR-synthesized image is measured precisely using mean squared error. In this way, areas around edges that are prone to synthesis artifacts are emphasized in the metric score. Two versions of morphological multiscale metric have been explored: (a) Morphological Pyramid Peak Signal-to-Noise Ratio metric (MP-PSNR) based on morphological pyramid decomposition, and (b) Morphological Wavelet Peak Signal-to-Noise Ratio metric (MW-PSNR) based on morphological wavelet decomposition. The performances of the proposed metrics have been tested using two databases which contain DIBR-synthesized images: the IRCCyN/IVC DIBR image database and MCL-3D stereoscopic image database. Proposed metrics achieve significantly higher correlation with human judgment compared to the state-of-the-art image quality metrics and compared to the tested metric dedicated to synthesis-related artifacts. The proposed metrics are computationally efficient given that the morphological operators involve only integer numbers and simple computations like min, max, and sum as well as simple calculation of MSE. MP-PSNR has slightly better performances than MW-PSNR. It has very good agreement with human judgment, Pearson’s 0.894, Spearman 0.77 when it is tested on the MCL-3D stereoscopic image database. We have demonstrated that PSNR has particularly good agreement with human judgment when it is calculated between images at higher scales of morphological multi-scale representations. Consequently, simplified and in essence reduced versions of multi-scale metrics are proposed, taking into account only detailed images at higher decomposition scales. The reduced version of MP-PSNR has very good agreement with human judgment, Pearson’s 0.904, Spearman 0.863 using IRCCyN/IVC DIBR image database.

Evaluating Image Quality of DIBR-synthesized Views in Texture and Depth Domains

Combining Local and Global Measures for DIBR-Synthesized Image Quality Evaluation.

Quality Index for View Synthesis by Measuring Instance Degradation and Global Appearance

An Efficient Depth Image-Based Rendering with Depth Reliability Maps for View Synthesis

Non-Subsampled Contourlet Transform and Ground-truth Score Generation based Quality Assessment for DIBR-Synthesized Views

Depth-image-based Rendering with Spatial and Temporal Texture Synthesis for 3DTV

Predicting the Quality of View Synthesis With Color-Depth Image Fusion

Quality Assessment for Virtual View Image Based on Edge Difference

Adaptive Structure and Texture Similarity Metric for Image Quality Assessment and Optimization

Non-reference Virtual View Quality Evaluation of MVD

Objective Quality Assessment of Depth Image Based Rendering in 3DTV System

DIBR-synthesized image quality assessment based on morphological multi-scale approach

Locally Adaptive Structure and Texture Similarity for Image Quality Assessment

Stereoscopic Image Quality Assessment Based on Depth and Texture Information

Blind CT Image Quality Assessment Using DDPM-derived Content and Transformer-based Evaluator

No-Reference Multi-Level Video Quality Assessment Metric for 3D-Synthesized Videos

Hybrid No-Reference Quality Metric for Singly and Multiply Distorted Images

Blind Image Quality Assessment Via Cross-View Consistency

No-Reference Quality Assessment for View Synthesis Using DoG-based Edge Statistics and Texture Naturalness

Image Quality Assessment Based on Multi-Feature Extraction and Synthesis with Support Vector Regression

Image Quality Assessment Using Full-Parameter Singular Value Decomposition