Abstract:Individuals with autism are known to face challenges with emotion regulation, and express their affective states in a variety of ways. With this in mind, an increasing amount of research on automatic affect recognition from speech and other modalities has recently been presented to assist and provide support, as well as to improve understanding of autistic individuals' behaviours. As well as the emotion expressed from the voice, for autistic children the dynamics of verbal speech can be inconsistent and vary greatly amongst individuals. The current contribution outlines a voice activity detection (VAD) system specifically adapted to autistic children's vocalisations. The presented VAD system is a recurrent neural network (RNN) with long short-term memory (LSTM) cells. It is trained on 130 acoustic Low-Level Descriptors (LLDs) extracted from more than 17 h of audio recordings, which were richly annotated by experts in terms of perceived emotion as well as occurrence and type of vocalisations. The data consist of 25 English-speaking autistic children undertaking a structured, partly robot-assisted emotion-training activity and was collected as part of the DE-ENIGMA project. The VAD system is further utilised as a preprocessing step for a continuous speech emotion recognition (SER) task aiming to minimise the effects of potential confounding information, such as noise, silence, or non-child vocalisation. Its impact on the SER performance is compared to the impact of other VAD systems, including a general VAD system trained from the same data set, an out-of-the-box Web Real-Time Communication (WebRTC) VAD system, as well as the expert annotations. Our experiments show that the child VAD system achieves a lower performance than our general VAD system, trained under identical conditions, as we obtain receiver operating characteristic area under the curve (ROC-AUC) metrics of 0.662 and 0.850, respectively. The SER results show varying performances across valence and arousal depending on the utilised VAD system with a maximum concordance correlation coefficient (CCC) of 0.263 and a minimum root mean square error (RMSE) of 0.107. Although the performance of the SER models is generally low, the child VAD system can lead to slightly improved results compared to other VAD systems and in particular the VAD-less baseline, supporting the hypothesised importance of child VAD systems in the discussed context.

Automatic Voice Classification Of Autistic Subjects

Classifying Autism from Crowdsourced Semi-Structured Speech Recordings: A Machine Learning Approach

Classifying Autism From Crowdsourced Semistructured Speech Recordings: Machine Learning Model Comparison Study

An Automated Assessment Framework for Atypical Prosody and Stereotyped Idiosyncratic Phrases Related to Autism Spectrum Disorder.

Detecting Autism Spectrum Disorders with Machine Learning Models Using Speech Transcripts

Automatic speaker diarization for natural conversation analysis in autism clinical trials

Exploring Speech Pattern Disorders in Autism using Machine Learning

Autism Speech Analysis using Acoustic Features

Automated speech- and text-based classification of neuropsychiatric conditions in a multidiagnostic setting

Automatically Predicting Perceived Conversation Quality in a Pediatric Sample Enriched for Autism

Enhancing Child Vocalization Classification with Phonetically-Tuned Embeddings for Assisting Autism Diagnosis

Autism Detection in Speech -- A Survey

Evaluating the Impact of Voice Activity Detection on Speech Emotion Recognition for Autistic Children

Speech and language patterns in autism: Towards natural language processing as a research and clinical tool

Developing an Artificial Intelligence Based Model for Autism Spectrum Disorder Detection in Children

Can Natural Speech Prosody Distinguish Autism Spectrum Disorders? A Meta-Analysis

Reliably quantifying the severity of social symptoms in children with autism using ASDSpeech

Automatic Classification of Adult Males With and Without Autism Spectrum Disorder by Non-contact Measurement of Autonomic Nervous System Activation

Automatic Classification of ASD Children Using Appearance-Based Features from Videos

A spectrogram image based intelligent technique for automatic detection of autism spectrum disorder from EEG

A Machine Learning Approach to the Diagnosis of Autism Spectrum Disorder and Multi-Systemic Developmental Disorder Based on Retrospective Data and ADOS-2 Score