Abstract:Existing RGB-D salient object detection (SOD) approaches concentrate on the cross-modal fusion between the RGB stream and the depth stream. They do not deeply explore the effect of the depth map itself. In this work, we design a single stream network to directly use the depth map to guide early fusion and middle fusion between RGB and depth, which saves the feature encoder of the depth stream and achieves a lightweight and real-time model. We tactfully utilize depth information from two perspectives: (1) Overcoming the incompatibility problem caused by the great difference between modalities, we build a single stream encoder to achieve the early fusion, which can take full advantage of ImageNet pre-trained backbone model to extract rich and discriminative features. (2) We design a novel depth-enhanced dual attention module (DEDA) to efficiently provide the fore-/back-ground branches with the spatially filtered features, which enables the decoder to optimally perform the middle fusion. Besides, we put forward a pyramidally attended feature extraction module (PAFE) to accurately localize the objects of different scales. Extensive experiments demonstrate that the proposed model performs favorably against most state-of-the-art methods under different evaluation metrics. Furthermore, this model is 55.5% lighter than the current lightest model and runs at a real-time speed of 32 FPS when processing a 384×384documentclass[12pt]{minimal}usepackage{amsmath}usepackage{wasysym}usepackage{amsfonts}usepackage{amssymb}usepackage{amsbsy}usepackage{mathrsfs}usepackage{upgreek}setlength{oddsidemargin}{-69pt}egin{document}$$384 imes 384$$end{document} image.

Salient Object Detection for RGB-D Image by Single Stream Recurrent Convolution Neural Network.

Depth Cue Enhancement and Guidance Network for RGB-D Salient Object Detection

Accurate salient object detection via dense recurrent connections and residual-based hierarchical feature integration.

Data-Level Recombination and Lightweight Fusion Scheme for RGB-D Salient Object Detection

Multi-level progressive parallel attention guided salient object detection for RGB-D images

RGBD Salient Object Detection, Based on Specific Object Imaging.

Specificity-preserving RGB-D Saliency Detection

A Single Stream Network for Robust and Real-Time RGB-D Salient Object Detection

Deeply-Supervised Recurrent Convolutional Neural Network for Saliency Detection.

Salient Object Detection for RGBD Video Via Spatial Interaction and Depth-Based Boundary Refinement

TranSal: Depth-guided Transformer for RGB-D Salient Object Detection

Boosting RGB-D Saliency Detection by Leveraging Unlabeled RGB Images

RGB-D Image Multi-Target Detection Method Based on 3D DSF R-CNN

Dynamic Selective Network for RGB-D Salient Object Detection

Triple-Complementary Network for RGB-D Salient Object Detection

Deep RGB-D Saliency Detection Without Depth

RGB-D Image Saliency Detection Based on Multi-modal Feature-fused Supervision

CNNs-Based RGB-D Saliency Detection Via Cross-View Transfer and Multiview Fusion.

Depth-induced Multi-scale Recurrent Attention Network for Saliency Detection

A Unified Structure for Efficient RGB and RGB-D Salient Object Detection

RGB-D Salient Object Detection via 3D Convolutional Neural Networks