Telepresence Video Quality Assessment

Zhenqiang Ying,Deepti Ghadiyaram,Alan Bovik

DOI: https://doi.org/10.48550/arXiv.2207.09956

2022-07-20

Abstract:Video conferencing, which includes both video and audio content, has contributed to dramatic increases in Internet traffic, as the COVID-19 pandemic forced millions of people to work and learn from home. Global Internet traffic of video conferencing has dramatically increased Because of this, efficient and accurate video quality tools are needed to monitor and perceptually optimize telepresence traffic streamed via Zoom, Webex, Meet, etc. However, existing models are limited in their prediction capabilities on multi-modal, live streaming telepresence content. Here we address the significant challenges of Telepresence Video Quality Assessment (TVQA) in several ways. First, we mitigated the dearth of subjectively labeled data by collecting ~2k telepresence videos from different countries, on which we crowdsourced ~80k subjective quality labels. Using this new resource, we created a first-of-a-kind online video quality prediction framework for live streaming, using a multi-modal learning framework with separate pathways to compute visual and audio quality predictions. Our all-in-one model is able to provide accurate quality predictions at the patch, frame, clip, and audiovisual levels. Our model achieves state-of-the-art performance on both existing quality databases and our new TVQA database, at a considerably lower computational expense, making it an attractive solution for mobile and embedded systems.

Computer Vision and Pattern Recognition,Image and Video Processing

What problem does this paper attempt to address?

The main problems that this paper attempts to solve are to improve the efficiency and accuracy of telepresence video quality assessment (TVQA). Specifically, the paper focuses on the following aspects:

1. **Integration of multi - modal signal quality models**: The paper proposes a new online video quality prediction framework that can handle video and audio signals and provide quality assessment from local to overall.

2. **Lack of subjective annotation data**: To overcome this challenge, the researchers collected approximately 2,000 telepresence videos from different countries and obtained approximately 80,000 subjective quality labels through crowdsourcing.

3. **Quality modeling of real - time streaming media content**: The method proposed in the paper can quickly provide quality feedback when the video stream arrives, which is suitable for real - time streaming media scenarios.

4. **Low computational cost**: The proposed model has low computational cost while maintaining high performance, and is suitable for application in mobile devices and embedded systems.

By constructing a large - scale subjective database and designing a multi - modal learning framework, the paper solves the above problems, thus providing a new solution for the quality assessment of telepresence videos.