Abstract:When recorders are used to survey acoustically conspicuous species, identification calls of the target species in recordings is essential for estimating density and abundance. We investigate how well deep neural networks identify vocalisations consisting of phrases of varying lengths, each containing a variable number of syllables. We use recordings of Hainan gibbon Nomascus hainanus vocalisations to develop and test the methods. We propose two methods for exploiting the two-level structure of such data. The first combines convolutional neural network (CNN) models with a hidden Markov model (HMM) and the second uses a convolutional recurrent neural network (CRNN). Both models learn acoustic features of syllables via a CNN and temporal correlations of syllables into phrases either via an HMM or recurrent network. We compare their performance to commonly used CNNs LeNet and VGGNet, and support vector machine (SVM). We also propose a dynamic programming method to evaluate how well phrases are predicted. This is useful for evaluating performance when vocalisations are labelled by phrases, not syllables. Our methods perform substantially better than the commonly used methods when applied to the gibbon acoustic recordings. The CRNN has an F-score of 90% on phrase prediction, which is 18% higher than the best of the SVM or LeNet and VGGNet methods. HMM post-processing raised the F-score of these last three methods to as much as 87%. The number of phrases is overestimated by CNNs and SVM, leading to error rates between 49% and 54%. With HMM, these error rates can be reduced to 0.4% at the lowest. Similarly, the error rate of CRNN's prediction is no more than 0.5%. CRNNs are better at identifying phrases of varying lengths composed of a varying number of syllables than simpler CNN or SVM models. We find a CRNN model to be best at this task, with a CNN combined with an HMM performing almost as well. We recommend that these kinds of models are used for species whose vocalisations are structured into phrases of varying lengths.

Pronunciation Erroneous Tendency Detection with Combination of Convolutional Neural Network and Long Short-Term Memory

Speech neuromuscular decoding based on spectrogram images using conformal predictors with Bi-LSTM.

Improve low-resource non-native mispronunciation detection with native speech by articulatory-based tandem feature

BiCAPT: Bidirectional Computer-Assisted Pronunciation Training with Normalizing Flows

Improve Mispronunciation Detection with Tandem Feature

End-to-end Mispronunciation Detection with Simulated Error Distance

Analysis on Mispronunciations in Capt Based on Computational Speech Perception

Masked Acoustic Unit for Mispronunciation Detection and Correction

An Application of Modified Confusion Network for Improving Mispronunciation Detection in Computer-aided Mandarin Pronunciation Training

Transducers with Pronunciation-aware Embeddings for Automatic Speech Recognition

Mispronunciation Detection with an Optimized Detection Network and Multi-Layer Perception Based Features

Grading the Severity of Mispronunciations in CAPT Based on Statistical Analysis and Computational Speech Perception

A two-stage mispronunciation detection approach for computer-assisted pronunciation training

Text-Aware End-to-end Mispronunciation Detection and Diagnosis

Phonological Level wav2vec2-based Mispronunciation Detection and Diagnosis Method

Improved Mispronunciation detection system using a hybrid CTC-ATT based approach for L2 English speakers

Applying Multitask Learning To Acoustic-Phonemic Model For Mispronunciation Detection And Diagnosis In L2 English Speech

Adaptive Frequency Cepstral Coefficients for Word Mispronunciation Detection

An Automatic Pronunciation Quality Assessing Algorithm for Computer Assisted Language Learning

Automatic lexical stress and pitch accent detection for L2 English speech using multi-distribution deep neural networks

Automated Call Detection for Acoustic Surveys with Structured Calls of Varying Length