Abstract:How to explore useful information from depth is the key success of the RGB-D saliency detection methods. While the RGB and depth images are from different domains, a modality gap will lead to unsatisfactory results for simple feature concatenation. Towards better performance, most methods focus on bridging this gap and designing different cross-modal fusion modules for features, while ignoring explicitly extracting some useful consistent information from them. To overcome this problem, we develop a simple yet effective RGB-D saliency detection method by learning discriminative cross-modality features based on the deep neural network. The proposed method first learns modality-specific features for RGB and depth inputs. And then we separately calculate the correlations of every pixel-pair in a cross-modality consistent way, i.e., the distribution ranges are consistent for the correlations calculated based on features extracted from RGB (RGB correlation) or depth inputs (depth correlation). From different perspectives, color or spatial, the RGB and depth correlations end up at the same point to depict how tightly each pixel-pair is related. Secondly, to complemently gather RGB and depth information, we propose a novel correlation-fusion to fuse RGB and depth correlations, resulting in a cross-modality correlation. Finally, the features are refined with both long-range cross-modality correlations and local depth correlations to predict salient maps. In which, the long-range cross-modality correlation provides context information for accurate localization, and the local depth correlation keeps good subtle structures for fine segmentation. In addition, a lightweight DepthNet is designed for efficient depth feature extraction. We solve the proposed network in an end-to-end manner. Both quantitative and qualitative experimental results demonstrate the proposed algorithm achieves favorable performance against state-of-the-art methods.

Discriminative Cross-Modal Transfer Learning and Densely Cross-Level Feedback Fusion for RGB-D Salient Object Detection

CNNs-Based RGB-D Saliency Detection Via Cross-View Transfer and Multiview Fusion.

CNN-Based RGB-D Salient Object Detection: Learn, Select, and Fuse

Double Cross-Modality Progressively Guided Network for RGB-D Salient Object Detection

Learning Discriminative Cross-Modality Features for RGB-D Saliency Detection

CIR-Net: Cross-Modality Interaction and Refinement for RGB-D Salient Object Detection

Cross-Modal Fusion and Progressive Decoding Network for RGB-D Salient Object Detection

Adaptive Fusion for RGB-D Salient Object Detection.

Cross-modal refined adjacent-guided network for RGB-D salient object detection

CMCLNet Cross-Modality Attention Fusion and Cross-Level Feature Interaction for RGBD salient object detection

RGB-D salient object detection via cross-modal joint feature extraction and low-bound fusion loss

Coordinate Attention Filtering Depth-Feature Guide Cross-Modal Fusion RGB-Depth Salient Object Detection

A Deep Multimodal Feature Learning Network for RGB-D Salient Object Detection

RGB-D Salient Object Detection Based on Cross-Modal and Cross-Level Feature Fusion

Transformer Fusion and Pixel-Level Contrastive Learning for RGB-D Salient Object Detection

Feature interaction and two-stage cross-modal fusion for RGB-D salient object detection

Discriminative feature fusion for RGB-D salient object detection

Modality-Induced Transfer-Fusion Network for RGB-D and RGB-T Salient Object Detection

RGBD Salient Object Detection via Disentangled Cross-modal Fusion

Lightweight Multi-Level Feature Difference Fusion Network for RGB-D-T Salient Object Detection

Learning Selective Mutual Attention and Contrast for RGB-D Saliency Detection