Abstract:In multi-lingual societies, where multiple languages are spoken in a small geographic vicinity, informal conversations often involve mix of languages. Existing speech technologies may be inefficient in extracting information from such conversations, where the speech data is rich in diversity with multiple languages and speakers. The DISPLACE (DIarization of SPeaker and LAnguage in Conversational Environments) challenge constitutes an open-call for evaluating and bench-marking the speaker and language diarization technologies on this challenging condition. The challenge entailed two tracks: Track-1 focused on speaker diarization (SD) in multilingual situations while, Track-2 addressed the language diarization (LD) in a multi-speaker scenario. Both the tracks were evaluated using the same underlying audio data. To facilitate this evaluation, a real-world dataset featuring multilingual, multi-speaker conversational far-field speech was recorded and distributed. Furthermore, a baseline system was made available for both SD and LD task which mimicked the state-of-art in these tasks. The challenge garnered a total of $42$ world-wide registrations and received a total of $19$ combined submissions for Track-1 and Track-2. This paper describes the challenge, details of the datasets, tasks, and the baseline system. Additionally, the paper provides a concise overview of the submitted systems in both tracks, with an emphasis given to the top performing systems. The paper also presents insights and future perspectives for SD and LD tasks, focusing on the key challenges that the systems need to overcome before wide-spread commercial deployment on such conversations.

Multilingual conversational telephony speech corpus creation for real world speaker diarization and recognition

Realistic multi-microphone data simulation for distant speech recognition

CNAMD Corpus: A Chinese Natural Audiovisual Multimodal Database of Conversations for Social Interactive Agents

Summary of the DISPLACE Challenge 2023 - DIarization of SPeaker and LAnguage in Conversational Environments

Advancing an Interdisciplinary Science of Conversation: Insights from a Large Multimodal Corpus of Human Speech

An Anechoic, High-Fidelity, Multidirectional Speech Corpus

Let's Go Real Talk: Spoken Dialogue Model for Face-to-Face Conversation

Dialogos: a robust system for human-machine spoken dialogue on the telephone

Enhancement And Analysis Of Conversational Speech: Jsalt 2017

Conversation Chronicles: Towards Diverse Temporal and Relational Dynamics in Multi-Session Conversations

Listening to Multi-talker Conversations: Modular and End-to-end Perspectives

The People's Speech: A Large-Scale Diverse English Speech Recognition Dataset for Commercial Usage

A Longitudinal Multi-modal Dataset for Dementia Monitoring and Diagnosis

Hindi-English Code-Switching Speech Corpus

Automatic Speech Recognition for Biomedical Data in Bengali Language

Creating a Corpus of Multilingual Parent-Child Speech Remotely: Lessons Learned in a Large-Scale Onscreen Picturebook Sharing Task

Speaker conditioned acoustic modeling for multi-speaker conversational ASR

Common Voice: A Massively-Multilingual Speech Corpus

Mere account mein kitna balance hai? -- On building voice enabled Banking Services for Multilingual Communities

3D-Speaker: A Large-Scale Multi-Device, Multi-Distance, and Multi-Dialect Corpus for Speech Representation Disentanglement