Abstract:We present a benchmark for detecting animal vocalizations in marine passive acoustic data including a published dataset, a concrete task and metrics to report. Three deep learning models for bioacoustics are tested on this benchmark and their performances are reported. Future efforts in the field should aim at the reduction of false‐positive rates in order to assure usability of the automated techniques. Passive acoustic monitoring (PAM) is commonly used to obtain year‐round continuous data on marine soundscapes harboring valuable information on species distributions or ecosystem dynamics. This continuously increasing amount of data requires highly efficient automated analysis techniques in order to exploit the full potential of the available data. Here, we propose a benchmark, which consists of a public dataset, a well‐defined task and evaluation procedure to develop and test automated analysis techniques. This benchmark focuses on the special case of detecting animal vocalizations in a real‐world dataset from the marine realm. We believe that such a benchmark is necessary to monitor the progress in the development of new detection algorithms in the field of marine bioacoustics. We ultimately use the proposed benchmark to test three detection approaches, namely ANIMAL‐SPOT, Koogu and a simple custom sequential convolutional neural network (CNN), and report performances. We report the performance of the three detection approaches in a blocked cross‐validation fashion with 11 site‐year blocks for a multi‐species detection scenario in a large marine passive acoustic dataset. Performance was measured with three simple metrics (i.e., true classification rate, noise misclassification rate and call misclassification rate) and one combined fitness metric, which allocates more weight to the minimization of false positives created by noise. Overall, ANIMAL‐SPOT performed the best with an average fitness metric of 0.6, followed by the custom CNN with an average fitness metric of 0.57 and finally Koogu with an average fitness metric of 0.42. The presented benchmark is an important step to advance in the automatic processing of the continuously growing amount of PAM data that are collected throughout the world's oceans. To ultimately achieve usability of developed algorithms, the focus of future work should be laid on the reduction of the false positives created by noise.

Detecting the presence of sperm whales echolocation clicks in noisy environments

Detecting the Presence of Sperm Whales' Echolocation Clicks in Noisy Environments

Click detection rate variability of central North Pacific sperm whales from passive acoustic towed arrays

Review of Cetacean's click detection algorithms

Deep Machine Learning Techniques for the Detection and Classification of Sperm Whale Bioacoustics

The acoustic behaviour of diving sperm whales observed with a hydrophone array

Automatic Detection and Annotation of Sperm Whale Codas

Bio-Inspired Covert Active Sonar Detection Method Based on the Encoding of Sperm Whale Clicks

Narwhal (Monodon monoceros) echolocation click rates to support cue counting passive acoustic density estimation

Click-click, who's there? Acoustically derived estimates of sperm whale size distribution off western Ireland

Compensating for the effects of site and equipment variation on delphinid species identification from their echolocation clicks

Heard off Heard: Acoustic detections of sperm whales (Physeter macrocephalus) and other cetaceans off Heard Island

3D tracking of foraging belugas from their clicks: Experiment from a coastal hydrophone array

Sperm whale acoustic abundance and dive behaviour in the western North Atlantic

Bioacoustical Periodic Pulse Train Signal Detection and Classification using Spectrogram Intensity Binarization and Energy Projection

Tracking sperm whales with a towed acoustic vector sensor

Spatial and Temporal Variations in the Occurrence and Foraging Activity of Coastal Dolphins in Menai Bay, Zanzibar, Tanzania

Sperm whale (Physeter macrocephalus) coda patterns in the Gulf of Catania, South-Western lonian Sea, Italy

Bayesian detection and tracking of odontocetes in 3-D from their echolocation clicks

Assessing Seasonality and Density From Passive Acoustic Monitoring of Signals Presumed to be From Pygmy and Dwarf Sperm Whales in the Gulf of Mexico

Deep learning in marine bioacoustics: a benchmark for baleen whale detection