Abstract:Spoken dialogue systems must be able to express empathy to achieve natural interaction with human users. However, laughter generation requires a high level of dialogue understanding. Thus, implementing laughter in existing systems, such as in conversational robots, has been challenging. As a first step toward solving this problem, rather than generating laughter from user dialogue, we focus on “shared laughter,” where a user laughs using either solo or speech laughs (initial laugh), and the system laughs in turn (response laugh). The proposed system consists of three models: 1) initial laugh detection, 2) shared laughter prediction, and 3) laugh type selection. We trained each model using a human-robot speed dating dialogue corpus. For the first model, a recurrent neural network was applied, and the detection performance achieved an F1 score of 82.6%. The second model used the acoustic and prosodic features of the initial laugh and achieved a prediction accuracy above that of the random prediction. The third model selects the type of system’s response laugh as social or mirthful laugh based on the same features of the initial laugh. We then implemented the full shared laughter generation system in an attentive listening dialogue system and conducted a dialogue listening experiment. The proposed system improved the impression of the dialogue system such as empathy perception compared to a naive baseline without laughter and a reactive system that always responded with only social laughs. We propose that our system can be used for situated robot interaction and also emphasize the need for integrating proper empathetic laughs into conversational robots and agents.

On Laughter and Speech-Laugh, Based on Observations of Child-Robot Interaction

Can a robot laugh with you?: Shared laughter generation for empathetic spoken dialogue

Happy or Evil Laughter? Analysing a Database of Natural Audio Samples

Executive Voiced Laughter and Social Approval: An Explorative Machine Learning Study

LaughNet: synthesizing laughter utterances from waveform silhouettes and a single laughter example

LaughTalk: Expressive 3D Talking Head Generation with Laughter

A generative framework for conversational laughter: Its 'language model' and laughter sound synthesis

Interactive robots for personalised multimodal comedy experiments

SMILE: Multimodal Dataset for Understanding Laughter in Video with Language Models

Analysis of Co-Laughter Gesture Relationship on RGB videos in Dyadic Conversation Contex

Laughter Synthesis: Combining Seq2seq modeling with Transfer Learning

An Analysis of Rhythmic Staccato-Vocalization Based on Frequency Demodulation for Laughter Detection in Conversational Meetings

Giving Robots a Voice: Human-in-the-Loop Voice Creation and open-ended Labeling

Laughter in everyday life: an event-based experience sampling method study using wrist-worn wearables

The Perception of Spontaneous and Volitional Laughter Across 21 Societies.

Ain't Misbehavin' -- Using LLMs to Generate Expressive Robot Behavior in Conversations with the Tabletop Robot Haru

Why So Serious? Exploring Humor in AAC Through AI-Powered Interfaces

Making Flow-Matching-Based Zero-Shot Text-to-Speech Laugh as You Like

Laughter Synthesis using Pseudo Phonetic Tokens with a Large-scale In-the-wild Laughter Corpus

Child-Robot Interactions for Second Language Tutoring to Preschool Children

Laughter and smiling facial expression modelling for the generation of virtual affective behavior