Abstract:The use of complementary information, namely depth or thermal information, has shown its benefits to salient object detection (SOD) during recent years. However, the RGB-D or RGB-T SOD problems are currently only solved independently, and most of them directly extract and fuse raw features from backbones. Such methods can he easily restricted by low-quality modality data and redundant cross-modal features. In this work, a unified end-to-end framework is designed to simultaneously analyze RCB-D and RGB-T SOD tasks. Specifically, to effectively tackle multi-modal features, we propose a novel multi-stage and multi-scale fusion network (MMNet), which consists of a cross-modal multi-stage fusion module (CMFM) and a bi-directional multi-scale decoder (BMD). Similar to the visual color stage doctrine in the human visual system (HVS), the proposed CMFM aims to explore important feature representations in feature response stage, and integrate them into cross-modal features in adversarial combination stage. Moreover, the proposed BMD learns the combination of multilevel cross-modal fused features to capture both local and global information of salient objects, and can further boost the multimodal SOD performance. The proposed unified cross-modality feature analysis framework based on two-stage and multi-scale information fusion can be used for diverse multi-modal SOD tasks. Comprehensive experiments (similar to 92K image-pairs) demonstrate that the proposed method consistently outperforms the other 21 state-of-the-art methods on nine benchmark datasets. This validates that our proposed method can work well on diverse multi-modal SOD tasks with good generalization and robustness, and provides a good multi-modal SOD benchmark.

MSRMNet: Multi-scale skip residual and multi-mixed features network for salient object detection

SLMSF-Net: A Semantic Localization and Multi-Scale Fusion Network for RGB-D Salient Object Detection

MRMNet: Multi-scale residual multi-branch neural network for object detection

MMNet: Multi-Stage and Multi-Scale Fusion Network for RGB-D Salient Object Detection

Multi-Scale and Detail-Enhanced Segment Anything Model for Salient Object Detection

M$^3$Net: Multilevel, Mixed and Multistage Attention Network for Salient Object Detection

Recurrent Multi-scale Transformer for High-Resolution Salient Object Detection

MSEDNet: Multi-scale fusion and edge-supervised network for RGB-T salient object detection

Adaptive Dual-Stream Sparse Transformer Network for Salient Object Detection in Optical Remote Sensing Images

Multi-Scale Cascade Network for Salient Object Detection.

MSF-YOLO: A multi-scale features fusion-based method for small object detection

Multi-scale and Multi-dimensional Weighted Network for Salient Object Detection in Optical Remote Sensing Images

A Multi-Source Feature Extraction Network for Salient Object Detection

Multi-scale Feature Aggregation Network for Salient Object Detection in Optical Remote Sensing Images

MFCINet: multi-level feature and context information fusion network for RGB-D salient object detection

Unified Information Fusion Network for Multi-Modal RGB-D and RGB-T Salient Object Detection

Multiscale and Multidimensional Weighted Network for Salient Object Detection in Optical Remote Sensing Images

LRMSNet: A New Lightweight Detection Algorithm for Multi-Scale SAR Objects

MFFNet: Multi-modal Feature Fusion Network for V-D-T Salient Object Detection

R2Net: Residual refinement network for salient object detection

One-stop multiscale reconciliation attention network with scribble supervision for salient object detection in optical remote sensing images