An ecological investigation of the effect of background noise on speech processing in a Virtual Classroom

Orel Levy,Adi Korisky,Yair Zvilichovsky,Elana Zion Golumbic
DOI: https://doi.org/10.1101/2024.01.05.574300
2024-01-05
Abstract:Many real-life situations can be extremely noisy. Psychoacoustic studies have shown that background noise can have a detrimental effect on the ability to process and understand speech. However, most studies use stimuli and task designs that are highly artificial, limiting their generalization to more realistic contexts. Moreover, to date, we do not fully understand the neurophysiological consequences of trying to pay attention to speech in a noisy place. To address this lab to real-life gap and increase the ecological validity of speech in noise research, here we introduce a novel audiovisual Virtual Reality (VR) experimental platform. Combined with neurophysiological measurements of neural activity (EEG), eye-gaze and skin conductance (GSR) we studied the effects of background noise in a realistic context where the ability to process and understand continuous speech is especially important: A VR Classroom. Participants (n=32) sat in a VR Classroom and were told to pay attention to mini-lecture segments by a virtual teacher. Trials were either Quiet or contained background construction noise, emitted from outside the classroom window, which was either Continuous (drilling) or Intermittent (air hammers). Result show that background noise had a detrimental effect on learning outcomes, which was also accompanied by reduced neural tracking of the teacher’s speech. Comparison of the two noise types showed that the intermittent construction noise was more disruptive than continuous noise, as index by both behavioral and neural measures, and it also elicited higher skin-conductance levels, reflecting heightened arousal. Interesting, eye-gaze dynamics were not affected by the presence of noise. This study advances our understanding of the neurophysiological effects of background noise and extends it to more ecologically relevant contexts. It also emphasizes the role that temporal dynamics play for processing speech in noise, highlighting the need to consider the features of realistic noises, as we expand speech in noise research to increasingly realistic circumstances.
Neuroscience
What problem does this paper attempt to address?
This paper discusses the impact of background noise on speech processing in virtual classrooms. The study used a virtual reality (VR) experimental platform that combined neural physiological measurements (such as EEG, eye tracking, and skin conductance) to improve the ecological validity of speech research in noisy environments. Participants listened to lectures from a virtual teacher in a VR classroom environment, and two different types of construction noise were added in some experiments: continuous drilling noise and intermittent pneumatic hammer noise. The results showed that background noise had a negative impact on learning outcomes and reduced neural tracking of teacher's speech. Intermittent noise was more disruptive than continuous noise, as reflected in behavioral and neural measurement indicators, and caused higher levels of skin conductance, indicating increased arousal. However, the presence of noise did not affect eye movement patterns. This study deepened our understanding of the neurophysiological effects of background noise and extended it to more realistic contexts, highlighting the importance of temporal dynamics of noise in speech processing and suggesting the need to consider more realistic noise characteristics.