Abstract:This paper proposes a methodology to assess video quality, spatial and social presence, empathy, attitude, and attention in 360-degree videos for immersive communications. The methodology is validated in an experiment which simulates an immersive communication environment where participants attend three conversations of different genre (everyday conversation, educational, and discussion) and from actor and observer acquisition perspectives. We consider three experimental conditions: (A) visualizing and rating the perceptual quality of contents in a Head-Mounted Display (HMD), (B) visualizing the contents in an HMD, and (C) visualizing the contents in an HMD where participants can see their hands and take notes. In all conditions participants visualize the same 360-degree videos, designed and acquired in the context of international experiences. Fifty-four participants were evenly distributed among A, B, and C conditions taking into account their international experience backgrounds (working or studying in a foreign country), obtaining a balanced and diverse sample of participants. In this paper, video quality is evaluated with Single-Stimulus Discrete Quality Evaluation (SSDQE) methodology. Spatial and social presence are evaluated with questionnaires adapted from the literature. Initial empathy is assessed with Interpersonal Reactivity Index (IRI) and a questionnaire is designed to evaluate the attitude after the visualization of each video. Attention is evaluated with three questions about the conversations of the contents that had pass/fail answers. The results from the subjective test validate the proposed methodology in immersive communications, showing that video quality experiments can be adapted to conditions imposed by experiments focused on the evaluation of socioemotional features in terms of contents of long-duration, different acquisition perspectives, and genre. In addition, the positive results related to the sense of social and spatial presence imply that technology can be relevant in the analyzed use case. Other main result is that the acquisition perspective greatly influences social presence. Finally, the annotated dataset, Student Experiences Around the World dataset (SEAW-dataset), obtained from the experiment is made publicly available for the research community.

Emerging Immersive Communication Systems: Overview, Taxonomy, and Good Practices for QoE Assessment

Emerging Immersive Communication Systems: Overview, Taxonomy, and Good Practises for QoE Assessment

From QoS to QoE: A Tutorial on Video Quality Assessment.

Methodology to Assess Quality, Presence, Empathy, Attitude, and Attention in 360-degree Videos for Immersive Communications

A Comprehensive Review of Quality of Experience for Emerging Video Services

A General QoE Assessment Framework for Applications and Services

Immersive media experience: a survey of existing methods and tools for human influential factors assessment

The Why, What, and How of Immersive Experience

QoE Assessment and Management of MV/3D Video Services

QoE Evaluation of Multimedia Services Based on Audiovisual Quality and User Interest.

Subjective Quality Assessment of One-to-One Video-Telephony Services.

Toward Immersive Experience: Evaluation for Interactive Network Services

Review of existing objective QoE methodologies

A Prospect of Interdisciplinary Methodology of QoE Assessment.

QoE in Video Transmission: A User Experience-Driven Strategy

QoE Evaluation Methods for 360-Degree VR Video Transmission

Subjective Evaluation of Visual Quality and Simulator Sickness of Short 360$^\circ$ Videos: ITU-T Rec. P.919

User Experience Evaluation in Shared Interactive Virtual Reality

QUALINET White Paper on Definitions of Immersive Media Experience (IMEx)

WebRTC-QoE: A Dataset of QoE Assessment of Subjective Scores, Network Impairments, and Facial & Speech Features

Questionnaires and Qualitative Feedback Methods to Measure User Experience in Mixed Reality