MarineDet: Towards Open-Marine Object Detection

Liang Haixin,Zheng Ziqiang,Ma Zeyu,Sai-Kit Yeung

2023-10-03

Abstract:Marine object detection has gained prominence in marine research, driven by the pressing need to unravel oceanic mysteries and enhance our understanding of invaluable marine ecosystems. There is a profound requirement to efficiently and accurately identify and localize diverse and unseen marine entities within underwater imagery. The open-marine object detection (OMOD for short) is required to detect diverse and unseen marine objects, performing categorization and localization simultaneously. To achieve OMOD, we present \textbf{MarineDet}. We formulate a joint visual-text semantic space through pre-training and then perform marine-specific training to achieve in-air-to-marine knowledge transfer. Considering there is no specific dataset designed for OMOD, we construct a \textbf{MarineDet dataset} consisting of 821 marine-relative object categories to promote and measure OMOD performance. The experimental results demonstrate the superior performance of MarineDet over existing generalist and specialist object detection algorithms. To the best of our knowledge, we are the first to present OMOD, which holds a more valuable and practical setting for marine ecosystem monitoring and management. Our research not only pushes the boundaries of marine understanding but also offers a standard pipeline for OMOD.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper primarily proposes a new method for object detection technology in marine environments, aiming to address the limitations present in current marine object detection algorithms. Specifically, the paper addresses the following key issues:

1. **The need for Open Vocabulary Marine Object Detection (OMOD)**: Traditional marine object detection algorithms are usually limited to known categories (close-set detection) and cannot effectively recognize or classify new or rare species. Therefore, this study proposes Open Vocabulary Marine Object Detection (OMOD), which can effectively identify both known and unknown marine entities.

2. **Building a large-scale marine object detection dataset**: To promote the development of OMOD, the authors constructed a dataset named MarineDet, which includes 821 categories of marine-related objects, covering a wide variety of marine biological and non-biological objects. These datasets not only help train more powerful detection models but also evaluate the performance of different algorithms on OMOD tasks.

3. **Proposing the MarineDet model**: The paper introduces a method called MarineDet, which pre-trains a joint visual-text semantic space and further conducts marine-specific training to achieve knowledge transfer from aerial to underwater. This approach enables the model to effectively detect various known and unknown marine objects without pre-defining all target categories.

4. **Addressing the challenges faced by OMOD**: The paper discusses the challenges faced by OMOD, including recognizing unseen or rare species and adapting to the constantly changing and diverse underwater environment. To address these issues, the MarineDet model optimizes its performance through contrastive learning and domain-specific fine-tuning.

5. **Experimental validation**: The authors conducted extensive experiments, including experiments under fully supervised settings and open vocabulary settings, to validate the effectiveness and boundaries of the proposed MarineDet method. The results show that MarineDet exhibits superior performance in multiple aspects, especially in detecting unseen marine object categories.

In summary, this paper aims to advance the field of marine ecosystem monitoring and management by proposing a new marine object detection framework, MarineDet, and constructing a large-scale dataset specifically for OMOD.