Abstract:In recent years, User Generated Content (UGC) has grown dramatically in video sharing applications. It is necessary for service-providers to use video quality assessment (VQA) to monitor and control users' Quality of Experience when watching UGC videos. However, most existing UGC VQA studies only focus on the visual distortions of videos, ignoring that the perceptual quality also depends on the accompanying audio signals. In this paper, we conduct a comprehensive study on UGC audio-visual quality assessment (AVQA) from both subjective and objective perspectives. Specially, we construct the first UGC AVQA database named SJTU-UAV database, which includes 520 in-the-wild UGC audio and video (A/V) sequences collected from the YFCC100m database. A subjective AVQA experiment is conducted on the database to obtain the mean opinion scores (MOSs) of the A/V sequences. To demonstrate the content diversity of the SJTU-UAV database, we give a detailed analysis of the SJTU-UAV database as well as other two synthetically-distorted AVQA databases and one authentically-distorted VQA database, from both the audio and video aspects. Then, to facilitate the development of AVQA fields, we construct a benchmark of AVQA models on the proposed SJTU-UAV database and other two AVQA databases, of which the benchmark models consist of AVQA models designed for synthetically distorted A/V sequences and AVQA models built through combining the popular VQA methods and audio features via support vector regressor (SVR). Finally, considering benchmark AVQA models perform poorly in assessing in-the-wild UGC videos, we further propose an effective AVQA model via jointly learning quality-aware audio and visual feature representations in the temporal domain, which is seldom investigated by existing AVQA models. Our proposed model outperforms the aforementioned benchmark AVQA models on the SJTU-UAV database and two synthetically distorted AVQA databases. The SJTU-UAV database and the c- de of the proposed model will be released to facilitate further research.

Non-reference audio quality assessment for online live music recordings

Attention-Guided Neural Networks for Full-Reference and No-Reference Audio-Visual Quality Assessment

Visual Quality Assessment for Web Videos

Perceiving Music Quality with GANs

An Improved Intelligent Machine Learning Approach to Music Recommendation Based on Big Data Techniques and DSO Algorithms

Quality-of-Experience Assessment for Ultra-low Latency Live Streaming Videos

Audio-Visual Quality Assessment for User Generated Content: Database and Method

Subjective and Objective Quality-of-Experience Evaluation Study for Live Video Streaming

A Brief Survey on Adaptive Video Streaming Quality Assessment

Subjective and Objective Audio-Visual Quality Assessment for User Generated Content

Automatic Organisation and Quality Analysis of User-Generated Content with Audio Fingerprinting

Video Background Music Generation: Dataset, Method and Evaluation

Sensitivity-Aware Spatial Quality Adaptation for Live Video Analytics

Perceptual Quality Assessment of Omnidirectional Audio-visual Signals

Assessment of Subjective and Objective Quality of Live Streaming Sports Videos

Towards a New Interface for Music Listening: A User Experience Study on YouTube

AutoMatch: A Large-scale Audio Beat Matching Benchmark for Boosting Deep Learning Assistant Video Editing

Exploration of Audio Quality Assessment and Anomaly Localisation Using Attention Models

A Comprehensive Survey for Evaluation Methodologies of AI-Generated Music

Amplifying the Music Listening Experience through Song Comments on Music Streaming Platforms

A Survey of Foundation Models for Music Understanding