Abstract:The task of salient object detection (SOD) often faces various challenges such as complex backgrounds and low appearance contrast. Depth information, which reflects the geometric shape of an object's surface, can be used as a supplement to visible information and receives increasing interest in SOD. However, depth sensors suffer from limited conditions and range (e.g., 4–5 ms at most in indoor scenes), and the imaging quality is usually low. We design a lightweight network in order to infer depth features while reducing computational complexities, which only needs a few parameters to effectively capture depth-specific features by fusing high-level features from the RGB modality. Both RGB features and inferred depth features might contain noises, and thus we design a fusion network, which includes a self-attention-based feature interaction module and a foreground-background enhancement module, to achieve an adaptive fusion of RGB and depth features. In addition, we introduce a multi-scale fusion module with different dilated convolutions to leverage useful local and global context clues. Experimental results on five benchmark datasets show that our approach significantly outperforms the state-of-the-art RGBD SOD methods, and also performs comparably against the state-of-the-art RGB SOD methods. The experimental results show that our multi-modal representation learning method can deal with the imaging limitations of single-modality data for RGB salient object detection, and the experimental results on multiple RGBD and RGB SOD datasets illustrate the effectiveness of our method.

Multi-modal Deep Feature Learning for RGB-D Object Detection

Large-Margin Multi-Modal Deep Learning for RGB-D Object Recognition

MMSS: Multi-modal Sharable and Specific Feature Learning for RGB-D Object Recognition.

Lightweight Multi-modal Representation Learning for RGB Salient Object Detection

Discriminative feature fusion for RGB-D salient object detection

Depth Cue Enhancement and Guidance Network for RGB-D Salient Object Detection

Deep RGB-D Saliency Detection with Depth-Sensitive Attention and Automatic Multi-Modal Fusion

Deep learning based object detection from multi-modal sensors: an overview

CNN-Based RGB-D Salient Object Detection: Learn, Select, and Fuse

Discriminative unimodal feature selection and fusion for RGB-D salient object detection

Cross-Modal Weighting Network for RGB-D Salient Object Detection

Object Detection Via Deeply Exploiting Depth Information

Correlated and Individual Multi-Modal Deep Learning for RGB-D Object Recognition.

Feature interaction and two-stage cross-modal fusion for RGB-D salient object detection

Multi-scale Cross-Modal Transformer Network for RGB-D Object Detection

RGB-D salient object detection via cross-modal joint feature extraction and low-bound fusion loss

Cross-Modal Attentional Context Learning for RGB-D Object Detection

Coordinate Attention Filtering Depth-Feature Guide Cross-Modal Fusion RGB-Depth Salient Object Detection

Exploiting Enhanced and Robust RGB-D Face Representation Via Progressive Multi-Modal Learning

Employing Bilinear Fusion and Saliency Prior Information for RGB-D Salient Object Detection

Learnable Depth-Sensitive Attention for Deep RGB-D Saliency Detection with Multi-modal Fusion Architecture Search