FathomVerse: A community science dataset for ocean animal discovery

Genevieve Patterson,Joost Daniels,Benjamin Woodward,Kevin Barnard,Giovanna Sainz,Lonny Lundsten,Kakani Katija

2024-12-03

Abstract:Can computer vision help us explore the ocean? The ultimate challenge for computer vision is to recognize any visual phenomena, more than only the objects and animals humans encounter in their terrestrial lives. Previous datasets have explored everyday objects and fine-grained categories humans see frequently. We present the FathomVerse v0 detection dataset to push the limits of our field by exploring animals that rarely come in contact with people in the deep sea. These animals present a novel vision challenge. The FathomVerse v0 dataset consists of 3843 images with 8092 bounding boxes from 12 distinct morphological groups recorded at two locations on the deep seafloor that are new to computer vision. It features visually perplexing scenarios such as an octopus intertwined with a sea star, and confounding categories like vampire squids and sea spiders. This dataset can push forward research on topics like fine-grained transfer learning, novel category discovery, species distribution modeling, and carbon cycle analysis, all of which are important to the care and husbandry of our planet.

Computer Vision and Pattern Recognition,Human-Computer Interaction

What problem does this paper attempt to address?

The main problems that this paper attempts to solve are the challenges of deep - sea biometric identification and discovery, which specifically include the following aspects:

1. **Insufficient exploration of marine organisms**: Currently, human exploration of the world's oceans is very limited. Only 7% of the ocean surface has long - term biological observation records. It is estimated that 30% to 60% of marine organisms are still unknown to the scientific community.

2. **Application of computer vision in the deep sea**: Existing computer vision data sets mainly focus on daily objects and fine - grained categories common to humans. However, deep - sea organisms, due to their rarity and complexity, pose new challenges to computer vision. For example, deep - sea organisms such as octopuses, starfish, vampire squids, and sea spiders have very different morphologies and behaviors from terrestrial organisms, which makes it difficult for traditional computer vision models to accurately identify these organisms.

3. **Difficulties in data annotation**: The professional time of marine biologists and taxonomists is precious. Annotating a large number of deep - sea images is both time - consuming and expensive. Therefore, a more efficient way is needed to generate annotated data to support model training and generalization to different geographical locations and concepts.

4. **Promoting the development of related research fields**: By creating a new data set containing deep - sea organisms, research in fields such as fine - grained transfer learning, new - class discovery, species distribution modeling, and carbon - cycle analysis can be promoted, which are crucial for the protection and management of the earth.

To this end, the paper proposes the FathomVerse v0 data set, a data set generated by a community science project. It aims to collect consensus annotations from global ocean enthusiasts in a gamified way, thereby helping scientists better understand and protect the deep - sea ecosystem. This data set contains 3,843 images and 8,092 bounding boxes, covering 12 different morphological groups and recording new scenes at two deep - sea locations.