Abstract:This paper proposes a methodology to assess video quality, spatial and social presence, empathy, attitude, and attention in 360-degree videos for immersive communications. The methodology is validated in an experiment which simulates an immersive communication environment where participants attend three conversations of different genre (everyday conversation, educational, and discussion) and from actor and observer acquisition perspectives. We consider three experimental conditions: (A) visualizing and rating the perceptual quality of contents in a Head-Mounted Display (HMD), (B) visualizing the contents in an HMD, and (C) visualizing the contents in an HMD where participants can see their hands and take notes. In all conditions participants visualize the same 360-degree videos, designed and acquired in the context of international experiences. Fifty-four participants were evenly distributed among A, B, and C conditions taking into account their international experience backgrounds (working or studying in a foreign country), obtaining a balanced and diverse sample of participants. In this paper, video quality is evaluated with Single-Stimulus Discrete Quality Evaluation (SSDQE) methodology. Spatial and social presence are evaluated with questionnaires adapted from the literature. Initial empathy is assessed with Interpersonal Reactivity Index (IRI) and a questionnaire is designed to evaluate the attitude after the visualization of each video. Attention is evaluated with three questions about the conversations of the contents that had pass/fail answers. The results from the subjective test validate the proposed methodology in immersive communications, showing that video quality experiments can be adapted to conditions imposed by experiments focused on the evaluation of socioemotional features in terms of contents of long-duration, different acquisition perspectives, and genre. In addition, the positive results related to the sense of social and spatial presence imply that technology can be relevant in the analyzed use case. Other main result is that the acquisition perspective greatly influences social presence. Finally, the annotated dataset, Student Experiences Around the World dataset (SEAW-dataset), obtained from the experiment is made publicly available for the research community.

Examining Multimodal Characteristics of Video to Understand User Engagement

Personalized User Engagement Modeling for Mobile Videos

Multimodal Engagement Analysis from Facial Videos in the Classroom

A multimodal approach for modeling engagement in conversation

Impacts of Presenting Extra Information in Short Videos Via Text and Voice on User Experience

Multimodal Content Analysis for Effective Advertisements on YouTube

Multimodal Sentiment Intensity Analysis in Videos: Facial Gestures and Verbal Messages

In Your Eyes: Modality Disentangling for Personality Analysis in Short Video

Methodology to Assess Quality, Presence, Empathy, Attitude, and Attention in 360-degree Videos for Immersive Communications

Towards an intelligent framework for multimodal affective data analysis

Characterizing Multimedia Information Environment through Multi-modal Clustering of YouTube Videos

Multimodal Characterization of Emotion within Multimedia Space

Analysing student engagement with 360-degree videos through multimodal data analytics and user annotations

Delving Deep into Engagement Prediction of Short Videos

Using Interaction Data to Predict Engagement with Interactive Media

IMPACT OF VISUAL MODALITIES IN MULTIMODAL PERSONALITY AND AFFECTIVE COMPUTING

Predicting Content Similarity Via Multimodal Modeling for Video-In-Video Advertising.

A Multimodal Sentiment Analysis Approach Based on a Joint Chained Interactive Attention Mechanism

An Estimation of Online Video User Engagement from Features of Continuous Emotions

Predicting Engagement in Video Lectures

Multimodal analysis within an interactive software environment: critical discourse perspectives