Abstract:One use of Autonomous Underwater Vehicles (AUVs) is the monitoring of habitats associated with threatened, endangered and protected marine species, such as the handfish of Tasmania, Australia. Seafloor imagery collected by AUVs can be used to identify individuals within their broader habitat context, but the sheer volume of imagery collected can overwhelm efforts to locate rare or cryptic individuals. Machine learning models can be used to identify the presence of a particular species in images using a trained object detector, but the lack of training examples reduces detection performance, particularly for rare species that may only have a small number of examples in the wild. In this paper, inspired by recent work in few-shot learning, images and annotations of common marine species are exploited to enhance the ability of the detector to identify rare and cryptic species. Annotated images of six common marine species are used in two ways. Firstly, the common species are used in a pre-training step to allow the backbone to create rich features for marine species. Secondly, a copy-paste operation is used with the common species images to augment the training data. While annotations for more common marine species are available in public datasets, they are often in point format, which is unsuitable for training an object detector. A popular semantic segmentation model efficiently generates bounding box annotations for training from the available point annotations. Our proposed framework is applied to AUV images of handfish, increasing average precision by up to 48\% compared to baseline object detection training. This approach can be applied to other objects with low numbers of annotations and promises to increase the ability to actively monitor threatened, endangered and protected species.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: **How to efficiently detect rare and endangered marine species in Autonomous Underwater Vehicle (AUV) images, especially when the number of training samples is very limited**. Specifically, the paper proposes solutions to the following challenges: 1. **Insufficient labeled data for rare species**: Many endangered species, such as handfish, are very scarce in the wild, so the labeled images available for training machine - learning models are also very limited. This leads to poor performance of traditional object - detection models, especially when dealing with rare species. 2. **Domain Shift problem**: Images collected by AUVs may come from different missions or use different devices, resulting in large conditional differences between training data and test data, thus affecting the generalization ability of the model. 3. **Small - object detection in complex backgrounds**: Species such as handfish are small in size and have secretive behaviors, and the backgrounds are complex, making manual annotation and detection very difficult. To solve these problems, the paper proposes a framework based on **few - shot learning** to enhance the detection ability of rare species by using a large amount of labeled data of common marine species. The main methods include: - **Pre - training**: Use the labeled data of common marine species to pre - train the backbone network of the detector to extract rich features. - **Data augmentation**: Paste instances of common species into images of rare species through the "copy - paste" operation to increase the diversity of training data and reduce domain shift. - **Point - annotation conversion**: Use the existing point - annotation data to generate bounding - box annotations for training object - detection models. The experimental results show that this framework significantly improves the Average Precision (AP) of handfish detection, especially in the case of fewer training samples, with a maximum improvement of 48%. This method is not only applicable to handfish, but can also be extended to the detection of other rare species, providing an effective tool for protecting endangered species.

Detecting Endangered Marine Species in Autonomous Underwater Vehicle Imagery Using Point Annotations and Few-Shot Learning