Audio-visual concert performances synchronize an audience's heart rates

Anna M. Czepiel,Lauren K. Fink,Mathias Scharinger,Christoph Seibert,Melanie Wald-Fuhrmann,Sonja A. Kotz

DOI: https://doi.org/10.1101/2024.04.10.588486

2024-09-28

Abstract:People enjoy engaging with music. Live music concerts provide an excellent option to investigate real-world music experiences, and at the same time, use neurophysiological synchrony to assess dynamic engagement. In the current study, we assessed engagement in a live concert setting using synchrony of cardiorespiratory measures, comparing inter-subject, stimulus-response, correlation, and phase coherence. As engagement might be enhanced in a concert setting by seeing musicians perform, we presented audiences with audio-only (AO) and audio-visual (AV) piano performances. Only correlation measures were above chance level. In comparing time-averaged synchrony across conditions, AV performances evoked higher inter-subject correlation of heart rate (ISC-HR). However, self-reported engagement did not correspond to synchrony when averaged across music pieces. On the other hand, time-resolved analyses show that synchronized deceleration-acceleration heart rate (HR) patterns, typical of an 'orienting response' (an index of directed attention), occurred within music pieces at salient events of section boundaries. That is, seeing musicians perform heightened audience engagement at structurally important moments in Western classical music. Overall, we could show that multisensory information shapes dynamic engagement. By comparing different synchrony measures, we further highlight the advantages of timeseries analysis, specifically ISC-HR, as a robust measure of holistic musical listening experiences in naturalistic concert settings.

Neuroscience

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to explore how audiovisual concerts can synchronize the heart rates of the audience. Specifically, the researchers wanted to assess the participants' engagement in the concert environment by measuring the synchrony of heart rate and respiration rate, and compared the performance under two different conditions: Audio - Only (AO) and Audio - Visual (AV). The research hypothesized that under the AV condition, the audience's heart rate synchrony would be higher, and this synchrony would be more significant at important moments in the music structure (such as section boundaries).

### Research Background and Objectives
- **Background**: People like to participate in music activities, especially in live concerts. Concerts provide an excellent scenario for exploring music experiences and can use neurophysiological synchrony to assess dynamic engagement.
- **Objective**: By assessing the synchrony of heart rate and respiration rate, the researchers aimed to explore whether the audience's engagement in music performances in the concert environment is affected by audiovisual information, and whether this effect is more pronounced at critical moments in the music structure.

### Research Methods
- **Participants**: 25 participants (excluding one participant with missing data), with an average age of 51.08 years and an average of 13 concerts attended per year.
- **Stimulus Materials**: Three Western classical piano pieces were selected, composed by Bach, Beethoven, and Messiaen respectively, which represent different musical styles and periods.
- **Experimental Design**: Participants watched the AO and AV versions of the performances in the concert respectively. After each piece, the participants rated their own level of engagement.
- **Data Collection**: Electrophysiological recording devices (such as ECG and respiratory belts) were used to continuously record the participants' heart rate and respiration rate.
- **Data Analysis**: The synchrony under different conditions was assessed by calculating the correlation and phase consistency of heart rate and respiration rate.

### Main Hypotheses
1. **Time - locked Response**: Verify whether the time - locked response of heart rate and respiration rate is significantly higher than the random level.
2. **Time - averaged Synchrony**: Under the AV condition, the time - averaged heart rate synchrony will be higher than that under the AO condition.
3. **Time - resolved Synchrony**: At important moments in the music structure (such as section boundaries), the time - resolved heart rate synchrony will be higher, especially under the AV condition.

### Conclusions
The study found that AV performances can indeed cause higher heart rate synchrony, especially at important moments in the music structure, indicating that visual information can enhance the audience's engagement at these critical moments. In addition, although the average of self - reported engagement did not differ significantly between different music segments, time - resolved analysis showed that the synchronous heart rate deceleration - acceleration pattern (a typical orienting response) occurred during important events in the music, further supporting the enhancing effect of visual information on audience engagement.

In summary, through multiple synchrony analysis methods, this study has demonstrated the role of multisensory information in shaping the dynamic music listening experience in the natural concert environment, especially by reflecting the audience's engagement through changes in heart rate synchrony.