Abstract:The neural radiance field (NeRF) representation has shown promising results in capturing 3D scenes. However, when it comes to editing or moving specific objects within virtual environments, it is essential to segment individual objects from the neural radiance field. Existing methods primarily rely on feature computation and clustering for 3D segmentation, but they often suffer from poor segmentation quality and low accuracy. Taking advantage of recent advancements in 2D image segmentation, we propose a 3D segmentation method that generates single-object segmentation from multiple viewpoint images. First, we compute camera poses using feature points from the original images. Then, with user guidance, we use the You Only Look Once (YOLO) network model to roughly locate the objects and utilize the resulting information as hints. Subsequently, we employ the Segment Anything Model (SAM) for fine-grained segmentation. The processed dataset is then rendered into a 3D segmented object using the neural radiance field, producing high-quality rendered images. By training on specific objects, we can segment object categories that the publicly available YOLO model cannot recognize, thereby enhancing generalization. Through experiments, we analyze the real-time performance of each module and qualitatively and quantitatively evaluate the effectiveness of our method. Compared to the N3F and ISRF methods, our approach achieves an average 17% improvement in Intersection over Union (IoU) and an average 9% improvement in Pixel Accuracy (Acc). We validate the performance of our method on the LLFF, mip-Nerf360 datasets, demonstrating its generalizability and effectiveness.

Segmentation And Recognition Of 3D Objects In Automatic Navigation

A New Approach of Point Cloud Processing and Scene Segmentation for Guiding the Visually Impaired

ObjectFusion: an Object Detection and Segmentation Framework with RGB-D SLAM and Convolutional Neural Networks

3D Human Landmark Detection and Segmentation with Semantic Knowledge

A novel segmentation algorithm for complex 3D mesh model in computer vision.

Underwater Object Recognition Using Point-Features, Bayesian Estimation and Semantic Information

Three-Dimensional Object Segmentation Method based on YOLO, SAM, and NeRF

Combined Edge- and Stixel-based Object Detection in 3D Point Cloud

An Approach for Construct Semantic Map with Scene Classification and Object Semantic Segmentation

3D-SeqMOS: A Novel Sequential 3D Moving Object Segmentation in Autonomous Driving

A Joint 3D-2D based Method for Free Space Detection on Roads

3D-Aware Object Goal Navigation Via Simultaneous Exploration and Identification

Fast Map Segmentation Method Based on Spectral Partition for Robot Semantic Navigation

Moving Object Segmentation in 3D LiDAR Data: A Learning-based Approach Exploiting Sequential Data

Automatic occlusion removal from 3D maps for maritime situational awareness

3-D Object Recognition Via Aspect Graph Aware 3-D Object Representation

Vehicle Recognition Based On Three-Dimensional Model

DMTN-Net: Semantic Segmentation Architecture for Surface Unmanned Vessels

Augmented reality navigation method based on image segmentation and sensor tracking registration technology

3D Object Recognition By Corresponding and Quantizing Neural 3D Scene Representations

SEMANTIC SEGMENTATION OF INDOOR 3D POINT CLOUDS BY JOINT OPTIMIZATION OF GEOMETRIC FEATURES AND NEURAL NETWORKS