Abstract:Spatio-temporal resolution adaptive (STRA) coding has been repeatedly proven to be a promising way to improve coding efficiency and reduce coding complexity. The wide consensus is that the optimal subsampled resolution and frame rate should be governed by so-called generalized rate-distortion performance based on the ultimately perceived distortion. However, it is non-trivial to accurately predict the quality of reconstructed videos due to the fact that the distortion originates from both subsampling and compression. To address this issue, we propose a novel video quality assessment model that is fully aware of the information available in downsampled videos for compression, such as resolution and frame rate. More specifically, the proposed model relies on quality-aware spatial features that are extracted by an image quality fine-tuned backbone. Subsequently, the spatio-temporal quality is modeled based on the transformer encoder, which is adaptive to the downsampling spatial and temporal resolutions. This enables the transformer encoder to produce discriminative features that capture long-range temporal dependencies related to the current context. The quality score, which is the output of the transformer encoder, thus reflects both the influence of the subsampling and compression. We conduct extensive experiments that demonstrate the superiority of the proposed model over state-of-the-art methods on four subsampling and compression video quality datasets. Furthermore, we apply the proposed model to bitrate ladder optimization, leading to a perceptual-aware spatial and temporal downsampling strategy that yields promising bitrate savings. The source codes of the proposed model will be publicly available at https://github.com/h4nwei/STRA-VQA.

Evaluate Mobile Video Quality in Hybrid Spatial and Temporal Domain

Efficient Video Quality Assessment Via 3d-Gradient Similarity Deviation

Spatio-temporal Ssim Index for Video Quality Assessment

Novel Spatio-Temporal Structural Information Based Video Quality Metric

Modeling the hybrid temporal and spatial resolutions effect for web video quality evaluation

A Spatiotemporal Model of Video Quality Assessment Via 3D Gradient Differencing.

Video Quality Assessment Metric Based On Spatio-Temporal Motion Information

Cross-Dimensional Quality Assessment For Low Bitrate Video

Encoder-Quantization-Motion-based Video Quality Metrics

A Video Quality Assessment Metric Based on Human Visual System

A Method of Video Quality Assessment Based on the Sensitive Region.

Spatiotemporal Statistics for Video Quality Assessment

Video Quality Assessment for Spatio-Temporal Resolution Adaptive Coding

Hybrid No-Reference Video Quality Assessment Focusing on Codec Effects.

Quality Assessment of Free-viewpoint Videos by Quantifying the Elastic Changes of Multi-Scale Motion Trajectories

Tile-Based Panoramic Video Quality Assessment

Stereoscopic video quality assessment model based on spatial-temporal structural information

Cross-Dimensional Perceptual Quality Assessment for Low Bit-Rate Videos

Visual Quality Assessment for Web Videos

Toward a Blind Quality Metric for Temporally Distorted Streaming Video.

Visual Saliency and Distortion Weighting Based Video Quality Assessment