Abstract:Recently, many models have been developed to predict video Quality of Experience (QoE), yet the applicability of these models still faces significant challenges. Firstly, many models rely on features that are unique to a specific dataset and thus lack the capability to generalize. Due to the intricate interactions among these features, a unified representation that is independent of datasets with different modalities is needed. Secondly, existing models often lack the configurability to perform both classification and regression tasks. Thirdly, the sample size of the available datasets to develop these models is often very small, and the impact of limited data on the performance of QoE models has not been adequately addressed. To address these issues, in this work we develop a novel and end-to-end framework termed as DeepQoE. The proposed framework first uses a combination of deep learning techniques, such as word embedding and 3D convolutional neural network (C3D), to extract generalized features. Next, these features are combined and fed into a neural network for representation learning. A learned representation will then serve as input for classification or regression tasks. We evaluate the performance of DeepQoE with three datasets. The results show that for small datasets (e.g., WHU-MVQoE2016 and Live-Netflix Video Database), the performance of state-of-the-art machine learning algorithms is greatly improved by using the QoE representation from DeepQoE (e.g., 35.71% to 44.82%); while for the large dataset (e.g., VideoSet), our DeepQoE framework achieves significant performance improvement in comparison to the best baseline method (90.94% vs. 82.84%). In addition to the much improved performance, DeepQoE has the flexibility to fit different datasets, to learn QoE representation, and to perform both classification and regression problems. We also develop a DeepQoE based adaptive bitrate streaming (ABR) system to verify that our framework can be easily applied to multimedia communication service. The software package of the DeepQoE framework has been released to facilitate the current research on QoE.

Learning QoE of Mobile Video Transmission with Deep Neural Network: A Data-Driven Approach

Deep Learning Based QoE Evaluation for Internet Video

Assessing the quality of experience in wireless networks for multimedia applications: A comprehensive analysis utilizing deep learning-based techniques

QoE Assessment Model Based on Continuous Deep Learning for Video in Wireless Networks

DeepQoE: A Multimodal Learning Framework for Video Quality of Experience (qoe) Prediction.

Predicting QoE of Video Streaming with Network-Context Features

Developing a QoE Monitoring Approach for Video Service Based on Mobile Terminals

DeepQoE: A Unified Framework for Learning to Predict Video QoE

Perceptual Quality Enhancement with Multi-scale Deep Learning for Video Transmission: A QoE Perspective

Quality Assessment for Networked Video Streaming Based on Deep Learning.

When Wireless Video Streaming Meets AI: A Deep Learning Approach

A Survey of Data-Driven Approach on Multimedia QoE Evaluation

Mining Mobile Intelligence for Wireless Systems: A Deep Neural Network Approach

No-Reference Qoe Prediction Model for Video Streaming Service in 3g Networks

QoS Prediction for Service Recommendation With Features Learning in Mobile Edge Computing Environment

Building a Large Dataset for Model-based QoE Prediction in the Mobile Environment

Deep Spatial and Temporal Information Based QoE Evaluation Model for HTTP Adaptive Streaming

Quality of Experience Evaluation Model with No-Reference VMAF Metric and Deep Spatio-temporal Features of Video

HAS QoE Prediction Based on Dynamic Video Features with Data Mining in LTE Network

Learning from Users: a Data-Driven Method of QoE Evaluation for Internet Video.

QoE Evaluation for Adaptive Video Streaming: Enhanced MDT with Deep Learning