Abstract:Several technological and scientific advances have been achieved recently in the fields of immersive systems (e.g., 360-degree/multiview video systems, augmented/mixed/virtual reality systems, immersive audio-haptic systems, etc.), which are offering new possibilities to applications and services in different communication domains, such as entertainment, virtual conferencing, working meetings, social relations, healthcare, and industry. Users of these immersive technologies can explore and experience the stimuli in a more interactive and personalized way than previous technologies (e.g., 2D video). Thus, considering the new technological challenges related to these systems and the new perceptual dimensions and interaction behaviors involved, a deep understanding of the users’ Quality of Experience (QoE) is required to satisfy their demands and expectations. In this sense, it is essential to foster the research on evaluating the QoE of immersive communication systems, since this will provide useful outcomes to optimize them and to identify the factors that can deteriorate the user experience. With this aim, subjective tests are usually performed following standard methodologies (e.g., ITU recommendations), which are designed for specific technologies and services. Although numerous user studies have been already published, there are no recommendations or standards that define common testing methodologies to be applied to evaluate immersive communication systems, such as those developed for images and video. Taking this into account, a revision of the QoE evaluation methods designed for previous technologies is required to develop robust and reliable methodologies for immersive communication systems. Thus, the objective of this paper is to provide an overview of existing immersive communication systems and related user studies, which can help on the definition of basic guidelines and testing methodologies to be used when performing user tests of immersive communication systems, such as 360-degree video-based telepresence, avatar-based social VR, cooperative AR, etc.

WebRTC-QoE: A Dataset of QoE Assessment of Subjective Scores, Network Impairments, and Facial & Speech Features

From QoS to QoE: A Tutorial on Video Quality Assessment.

A General QoE Assessment Framework for Applications and Services

Subjective Quality Assessment of One-to-One Video-Telephony Services.

Enabling QoE Learning and Prediction of WebRTC Video Communication in WiFi Networks

QoE Evaluation of Multimedia Services Based on Audiovisual Quality and User Interest.

Estimating WebRTC Video QoE Metrics Without Using Application Headers

SpeechQoE

Subjective and Objective Quality-of-Experience Evaluation Study for Live Video Streaming

Quality of Experience Evaluation for Networked Video Services

An Estimated QoE Model for Video Telephone Service.

Machine Learning-Driven Open-Source Framework for Assessing QoE in Multimedia Networks

Measurements and Applications of QoE for Multimedia Communications

Emerging Immersive Communication Systems: Overview, Taxonomy, and Good Practises for QoE Assessment

Emerging Immersive Communication Systems: Overview, Taxonomy, and Good Practices for QoE Assessment

QoE Based WebRTC Transmission Optimization Strategy for Mobile Network

Subjective and Objective Quality-of-Experience Assessment for 3D Talking Heads

An Affect-based Approach for QoE evaluation in VoIP Systems

Multimedia Communication Quality Assessment Testbeds

An Objective Multi-Layer Qoe Evaluation For Tcp Video Streaming

QoE Evaluation for Adaptive Video Streaming: Enhanced MDT with Deep Learning