Abstract:Enabled by advancing technology, coral reef researchers increasingly prefer use of image-based surveys over approaches depending solely upon in situ observations, interpretations, and recordings of divers. The images collected, and derivative products such as orthographic projections and 3D models, allow researchers to study a comprehensive digital twin of their field sites. Spatio-temporally located twins can be compared and annotated, enabling researchers to virtually return to sites long after they have left them. While these new data expand the variety and specificity of biological investigation that can be pursued, they have introduced the much-discussed Big Data Problem: research labs lack the human and computational resources required to process and analyze imagery at the rate it can be collected. The rapid development of unmanned underwater vehicles suggests researchers will soon have access to an even greater volume of imagery and other sensor measurements than can be collected by diver-piloted platforms, further exacerbating data handling limitations. Thoroughly segmenting (tracing the extent of and taxonomically identifying) organisms enables researchers to extract the information image products contain, but is very time-consuming. Analytic techniques driven by neural networks offer the possibility that the segmentation process can be greatly accelerated through automation. In this study, we examine the efficacy of automated segmentation on three different image-derived data products: 3D models, and 2D and 2.5D orthographic projections thereof; we also contrast their relative accessibility and utility to different avenues of biological inquiry. The variety of network architectures and parameters tested performed similarly, ∼80% IoU for the genus Porites , suggesting that the primary limitations to an automated workflow are 1) the current capabilities of neural network technology, and 2) consistency and quality control in image product collection and human training/testing dataset generation.

Introducing SPROUT (Semi-automated Parcellation of Region Outputs Using Thresholding): an adaptable computer vision tool to generate 3D segmentations

GrowSP: Unsupervised Semantic Segmentation of 3D Point Clouds

3D Object Segmentation Using Cross-Window Point Transformer with Latent Semantic Boundary Guidance

Spatio-Temporal Video Segmentation of Static Scenes and Its Applications

SA3DIP: Segment Any 3D Instance with Potential 3D Priors

AGILE3D: Attention Guided Interactive Multi-object 3D Segmentation

Point-SAM: Promptable 3D Segmentation Model for Point Clouds

AutoProSAM: Automated Prompting SAM for 3D Multi-Organ Segmentation

SproutAngio: an open-source bioimage informatics tool for quantitative analysis of sprouting angiogenesis and lumen space

DCSEG: Decoupled 3D Open-Set Segmentation using Gaussian Splatting

SEGMENT3D: A Web-based Application for Collaborative Segmentation of 3D images used in the Shoot Apical Meristem

Automated 2D, 2.5D, and 3D Segmentation of Coral Reef Pointclouds and Orthoprojections

Interactive Object Segmentation in 3D Point Clouds

SuRVoS: Super-Region Volume Segmentation Workbench

SASFormer: Transformers for Sparsely Annotated Semantic Segmentation.

Shape Prior Generation and Geodesic Active Contour Interactive Iterating Algorithm (SPACIAL): Fully Automatic Segmentation for 3D Lumen in Intravascular Optical Coherence Tomography Images.

An effective and open source interactive 3D medical image segmentation solution

SAMPro3D: Locating SAM Prompts in 3D for Zero-Shot Scene Segmentation

A Fast Method for 3D Object Segmentation Using Sparse Point Clouds

Eff-3DPSeg: 3D organ-level plant shoot segmentation using annotation-efficient point clouds

PropSAM: A Propagation-Based Model for Segmenting Any 3D Objects in Multi-Modal Medical Images