Abstract:Background Loneliness is a growing public health problem that has been exacerbated in vulnerable groups during the COVID-19 pandemic. Social support interventions have been shown to reduce loneliness, including when delivered through technology. Digital humans are a new type of computer agent that show promise as supportive peers in health care. For digital humans to be effective and engaging support persons, it is important that they develop closeness with people. Closeness can be increased by emotional expressiveness, particularly in female relationships. However, it is unknown whether emotional expressiveness improves relationships with digital humans and affects physiological responses. Objective The aim of this study is to investigate whether emotional expression by a digital human can affect psychological and physiological outcomes and whether the effects are moderated by the user’s gender. Methods A community sample of 198 adults (101 women, 95 men, and 2 gender-diverse individuals) was block-randomized by gender to complete a 15-minute self-disclosure conversation with a female digital human in 1 of 6 conditions. In these conditions, the digital human varied in modality richness and emotional expression on the face and in the voice (emotional, neutral, or no face; emotional or neutral voice). Perceived loneliness, closeness, social support, caring perceptions, and stress were measured after each interaction. Heart rate, skin temperature, and electrodermal activity were assessed during each interaction. 3-way factorial analyses of variance with post hoc tests were conducted. Results Emotional expression in the voice was associated with greater perceptions of caring and physiological arousal during the interaction, and unexpectedly, with lower feelings of support. User gender moderated the effect of emotional expressiveness on several outcomes. For women, an emotional voice was associated with increased closeness, social support, and caring perceptions, whereas for men, a neutral voice increased these outcomes. For women, interacting with a neutral face was associated with lower loneliness and subjective stress compared with no face. Interacting with no face (ie, a voice-only black screen) resulted in lower loneliness and subjective stress for men, compared with a neutral or emotional face. No significant results were found for heart rate or skin temperature. However, average electrodermal activity was significantly higher for men while interacting with an emotional voice. Conclusions Emotional expressiveness in a female digital human has different effects on loneliness, social, and physiological outcomes for men and women. The results inform the design of digital human support persons and have theoretical implications. Further research is needed to evaluate how more pronounced emotional facial expressions in a digital human might affect the results. Trial Registration Australia New Zealand Clinical Trials Registry (ANZCTR) ACTRN12621000865819; https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=381816&isReview

The Effect of Expressive Biosignals on Empathy and Closeness for a Stigmatized Group Member

Discrete Emotions Discovered by Contactless Measurement of Facial Blood Flows

The Influence of Emotional State on Empathic Accuracy

Mood Congruency Affects Physiological Synchrony but Not Empathic Accuracy in a Naturalistic Empathy Task.

Functional Graph Contrastive Learning of Hyperscanning EEG Reveals Emotional Contagion Evoked by Stereotype-Based Stressors

Multimodal Affective State Assessment Using fNIRS + EEG and Spontaneous Facial Expression

Feedback matters: EEG correlates of empathy involved in the naturalistic communication of emotions

Assessment of Empathy in an Affective VR Environment using EEG Signals

Significant Otter: Understanding the Role of Biosignals in Communication

Effects of an empathy enhancement program using patient stories on attitudes and stigma toward mental illness among nursing students

How self-disclosure of negative experiences shapes prosociality?

Emotional Empathy in the Social Regulation of Distress: A Dyadic Approach

The physiological measurement and evaluation of empathy of video content

The effect of modification of expressive displays on vicarious emotional arousal

Effects of Emotional Expressiveness of a Female Digital Human on Loneliness, Stress, Perceived Support, and Closeness Across Genders: Randomized Controlled Trial

The Resting-State Electroencephalogram Microstate Correlations With Empathy and Their Moderating Effect on the Relationship Between Empathy and Disgust

Your blush gives you away: detecting hidden mental states with remote photoplethysmography and thermal imaging

Breast cancer survivor testimonies: Effects of narrative and emotional valence on affect and cognition

Social Interactions and Cardiovascular Reactivity during Problem Disclosure among Friends

Sharing Happy Stories Increases Interpersonal Closeness: Interpersonal Brain Synchronization as a Neural Indicator

Signatures of social pain empathy: general and process-specific brain-wide representations of social exclusion and separation