Abstract:The use of RGB-D information for salient object detection (SOD) has been extensively explored in recent years. However, relatively few efforts have been put toward modeling SOD in real-world human activity scenes with RGB-D. In this article, we fill the gap by making the following contributions to RGB-D SOD: 1) we carefully collect a new Salient Person (SIP) data set that consists of ~1 K high-resolution images that cover diverse real-world scenes from various viewpoints, poses, occlusions, illuminations, and background s; 2) we conduct a large-scale (and, so far, the most comprehensive) benchmark comparing contemporary methods, which has long been missing in the field and can serve as a baseline for future research, and we systematically summarize 32 popular models and evaluate 18 parts of 32 models on seven data sets containing a total of about 97k images; and 3) we propose a simple general architecture, called deep depth-depurator network (D3Net). It consists of a depth depurator unit (DDU) and a three-stream feature learning module (FLM), which performs low-quality depth map filtering and cross-modal feature learning, respectively. These components form a nested structure and are elaborately designed to be learned jointly. D3Net exceeds the performance of any prior contenders across all five metrics under consideration, thus serving as a strong model to advance research in this field. We also demonstrate that D3Net can be used to efficiently extract salient object masks from real scenes, enabling effective background-changing application with a speed of 65 frames/s on a single GPU. All the saliency maps, our new SIP data set, the D3Net model, and the evaluation tools are publicly available at https://github.com/DengPingFan/D3NetBenchmark.

SUN RGB-D: A RGB-D scene understanding benchmark suite

Benchmarking Large-Scale Multi-View 3D Reconstruction Using Realistic Synthetic Images

SSCBench: A Large-Scale 3D Semantic Scene Completion Benchmark for Autonomous Driving

Matterport3D: Learning from RGB-D Data in Indoor Environments

SSCBench: Monocular 3D Semantic Scene Completion Benchmark in Street Views

Rethinking RGB-D Salient Object Detection: Models, Data Sets, and Large-Scale Benchmarks

ScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes

RGB-D salient object detection: A survey

KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D

NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding

3DSRBench: A Comprehensive 3D Spatial Reasoning Benchmark

DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Vision

PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understanding

3DRef: 3D Dataset and Benchmark for Reflection Detection in RGB and Lidar Data

UAV3D: A Large-scale 3D Perception Benchmark for Unmanned Aerial Vehicles

Holistic Understanding of 3D Scenes as Universal Scene Description

Indoor Scene Understanding with RGB-D Images: Bottom-up Segmentation, Object Detection and Semantic Segmentation

SUN database: Large-scale scene recognition from abbey to zoo

ClearPose: Large-scale Transparent Object Dataset and Benchmark