Abstract:The sudden outbreak of COVID-19 has resulted in tough challenges for the field of biometrics due to its spread via physical contact, and the regulations of wearing face masks. Given these constraints, voice biometrics can offer a suitable contact-less biometric solution; they can benefit from models that classify whether a speaker is wearing a mask or not. This article reviews the Mask Sub-Challenge (MSC) of the INTERSPEECH 2020 COMputational PARalinguistics challengE (ComParE), which focused on the following classification task: Given an audio chunk of a speaker, classify whether the speaker is wearing a mask or not. First, we report the collection of the Mask Augsburg Speech Corpus (MASC) and the baseline approaches used to solve the problem, achieving a performance of 71.8 % Unweighted Average Recall (UAR). We then summarise the methodologies explored in the submitted and accepted papers that mainly used two common patterns: (i) phonetic-based audio features, or (ii) spectrogram representations of audio combined with Convolutional Neural Networks (CNNs) typically used in image processing. Most approaches enhance their models by adapting ensembles of different models and attempting to increase the size of the training data using various techniques. We review and discuss the results of the participants of this sub-challenge, where the winner scored a UAR of 80.1 % . Moreover, we present the results of fusing the approaches, leading to a UAR of 82.6 % . Finally, we present a smartphone app that can be used as a proof of concept demonstration to detect in real-time whether users are wearing a face mask; we also benchmark the run-time of the best models.

The ACM Multimedia 2022 Computational Paralinguistics Challenge: Vocalisations, Stuttering, Activity, & Mosquitoes

The INTERSPEECH 2021 Computational Paralinguistics Challenge: COVID-19 Cough, COVID-19 Speech, Escalation & Primates

The ACM Multimedia 2023 Computational Paralinguistics Challenge: Emotion Share & Requests

MEGC2024: ACM Multimedia 2024 Facial Micro-Expression Grand Challenge

IEEE SLT 2021 Alpha-mini Speech Challenge: Open Datasets, Tracks, Rules and Baselines

ASVspoof 2021: Towards Spoofed and Deepfake Speech Detection in the Wild

MEGC2023: ACM Multimedia 2023 ME Grand Challenge

Advancing Audio Emotion and Intent Recognition with Large Pre-Trained Models and Bayesian Inference

The First Multimodal Information Based Speech Processing (Misp) Challenge: Data, Tasks, Baselines And Results

The Multimodal Information based Speech Processing (MISP) 2022 Challenge: Audio-Visual Diarization and Recognition

Mask Detection and Breath Monitoring from Speech: on Data Augmentation, Feature Representation and Modeling

The Interspeech 2024 Challenge on Speech Processing Using Discrete Units

Summary on the Multimodal Information Based Speech Processing (MISP) 2022 Challenge.

The Multimodal Information Based Speech Processing (MISP) 2023 Challenge: Audio-Visual Target Speaker Extraction

Aalto's End-to-End DNN systems for the INTERSPEECH 2020 Computational Paralinguistics Challenge

Face mask recognition from audio: The MASC database and an overview on the mask challenge

ASVspoof 5: Crowdsourced Speech Data, Deepfakes, and Adversarial Attacks at Scale

Applying Speech Tempo-Derived Features, BoAW and Fisher Vectors to Detect Elderly Emotion and Speech in Surgical Masks

Audio-Visual Speech Recognition in MISP2021 Challenge: Dataset Release and Deep Analysis

BCN2BRNO: ASR System Fusion for Albayzin 2020 Speech to Text Challenge