Abstract:We present a benchmark for detecting animal vocalizations in marine passive acoustic data including a published dataset, a concrete task and metrics to report. Three deep learning models for bioacoustics are tested on this benchmark and their performances are reported. Future efforts in the field should aim at the reduction of false‐positive rates in order to assure usability of the automated techniques. Passive acoustic monitoring (PAM) is commonly used to obtain year‐round continuous data on marine soundscapes harboring valuable information on species distributions or ecosystem dynamics. This continuously increasing amount of data requires highly efficient automated analysis techniques in order to exploit the full potential of the available data. Here, we propose a benchmark, which consists of a public dataset, a well‐defined task and evaluation procedure to develop and test automated analysis techniques. This benchmark focuses on the special case of detecting animal vocalizations in a real‐world dataset from the marine realm. We believe that such a benchmark is necessary to monitor the progress in the development of new detection algorithms in the field of marine bioacoustics. We ultimately use the proposed benchmark to test three detection approaches, namely ANIMAL‐SPOT, Koogu and a simple custom sequential convolutional neural network (CNN), and report performances. We report the performance of the three detection approaches in a blocked cross‐validation fashion with 11 site‐year blocks for a multi‐species detection scenario in a large marine passive acoustic dataset. Performance was measured with three simple metrics (i.e., true classification rate, noise misclassification rate and call misclassification rate) and one combined fitness metric, which allocates more weight to the minimization of false positives created by noise. Overall, ANIMAL‐SPOT performed the best with an average fitness metric of 0.6, followed by the custom CNN with an average fitness metric of 0.57 and finally Koogu with an average fitness metric of 0.42. The presented benchmark is an important step to advance in the automatic processing of the continuously growing amount of PAM data that are collected throughout the world's oceans. To ultimately achieve usability of developed algorithms, the focus of future work should be laid on the reduction of the false positives created by noise.

Entropy-Based Automatic Detection of Marine Mammal Tonal Calls

Soft-Output Signal Detection for Cetacean Vocalizations Using Spectral Entropy, K-Means Clustering and the Continuous Wavelet Transform

Analyzing and Classifying Fish Vocalization in the East China Sea Coastal Area

On Marine Mammal Acoustic Detection Performance Bounds

Development of deep neural networks for marine mammal call detection using an open-source, user friendly tool

Adaptive Detection Method for Dual Threshold Broadband Underwater Acoustic Signals Based on Morphological Filtering

Underwater multi-target passive detection based on transient signals using adaptive empirical mode decomposition

Application to the Call of Southern Right Whale

Deep learning in marine bioacoustics: a benchmark for baleen whale detection

Adaptive Weighted Envelope Spectrum: A Robust Spectral Quantity for Passive Acoustic Detection of Underwater Propeller Based on Spectral Coherence

Detection and Recognition of Underwater Acoustic Communication Signal Under Ocean Background Noise

Research Advance in Underwater Detection of Marine Status for Underwater Vehicles

Passive ship detection and classification using hybrid cepstrums and deep compound autoencoders

Underwater Acoustic Signal Detection Using Calibrated Hidden Markov Model with Multiple Measurements

Application of Wavelet Transform for the Detection of Cetacean Acoustic Signals

An archive of odontocete sounds for automatic call recognition

Combining Denoising Autoencoders and Dynamic Programming for Acoustic Detection and Tracking of Underwater Moving Targets

Long-Range Automatic Detection, Acoustic Signature Characterization and Bearing-Time Estimation of Multiple Ships with Coherent Hydrophone Array

A classification method of marine mammal calls based on two-channel fusion network

Fish sounds: towards the evaluation of marine acoustic biodiversity through data-driven audio source separation

A Comparative Study of Multiscale Sample Entropy and Hierarchical Entropy and Its Application in Feature Extraction for Ship-Radiated Noise