Abstract:Many popular video quality assessment (VQA) methods usually build models by simulating the process of human visual perception and adopt a simple regression strategy to predict video quality scores. However, these methods either hardly pay enough attention to regression processing prone to misprediction, or fail to accurately understand video content containing changes of movement or sudden movements. To remedy these, we propose a full reference (FR) video quality assessment model that integrates multi-task learning regression and analysis of spatio-temporal features to conduct video quality predictions. Firstly, the model arranges each frame of the reference and distorted videos into patches and calculates their entropy values to guide the selection of frame patches. A 2D Siamese network is then applied on the selected patches to learn spatial information. To more effectively capture temporal distortions, a multi-frame difference map is computed on each distorted video. The computed multi-frame difference maps are also partitioned into patches to select half of the ones with highest entropy values as temporal features. Additionally, we incorporate the temporal masking effect to optimize the spatial error and temporal features and adopt 3D convolutional neural network (CNN) in spatio-temporal feature fusion. Following recent evidence towards quality classification and quality regression, a constrained multi-task learning regression model is designed to aggregate the quality score, using quality classification subtask to contrain and optimize quality regression main task. Finally, the video quality score is predicted through the regression branch. We have evaluated our algorithm on five public VQA databases. The experimental results have revealed that the proposed algorithm can achieve superior performance as compared with the existing VQA methods.

Distortion-Aware Mutual Constraint for Screen Content Image Quality Assessment.

A Robust Evaluation Method for Motion Distortion of TDICCD Image

Encoding Distortions for Multi-task Full-Reference Image Quality Assessment

McmIQA: Multi-Module Collaborative Model for No-Reference Image Quality Assessment

Objective quality assessment for image retargeting based on hybrid distortion pooled model

Systemic distortion analysis with deep distortion directed image quality assessment models

Staged-Learning: Assessing the Quality of Screen Content Images from Distortion Information

Image quality assessment based on geometric structural distortion model

MSPP-IQA: Adaptive Blind Image Quality Assessment Based on Multi-level Spatial Pyramid Pooling.

Image Quality Assessment Based on Local Linear Information and Distortion-Specific Compensation

From Distortion Manifold to Perceptual Quality: a Data Efficient Blind Image Quality Assessment Approach

Multi-Task Rank Learning for Image Quality Assessment

Image Quality Assessment Based on Joint Quality-Aware Representation Construction in Multiple Domains

A Full-Reference Image Quality Assessment Method Via Deep Meta-Learning and Conformer

Learning Hybrid Representations of Semantics and Distortion for Blind Image Quality Assessment

Image quality assessment based on local information distortion modeling

Distortion Recognition for Image Quality Assessment with Convolutional Neural Network

Multi-Metric Fusion Network for Image Quality Assessment

Deep Video Quality Assessment Using Constrained Multi-Task Regression and Spatio-temporal Feature Fusion.

Visual Interaction Perceptual Network for Blind Image Quality Assessment

A Multi-Scale Learning Local Phase and Amplitude Blind Image Quality Assessment for Multiply Distorted Images