Abstract:A modality separate tri‐stream net is proposed, which fully explored the potential of shared features and specific features between different modalities, and completed the clear separation and extraction of the two features of different modalities, thereby improving the utilisation of the multi‐modal data. Most of the existing RGB‐T salient object detection methods are usually based on dual‐stream encoding single‐stream decoding network architecture. These models always rely on the quality of fusion features, which often focus on modality‐shared features and overlook modality‐specific features, thus failing to fully utilise the rich information contained in multi‐modality data. To this end, a modality separate tri‐stream net (MSTNet), which consists of a tri‐stream encoding (TSE) structure and a tri‐stream decoding (TSD) structure is proposed. The TSE explicitly separates and extracts the modality‐shared and modality‐specific features to improve the utilisation of multi‐modality data. In addition, based on the hybrid‐attention and cross‐attention mechanism, we design an enhanced complementary fusion module (ECF), which fully considers the complementarity between the features to be fused and realises high‐quality feature fusion. Furthermore, in TSD, the quality of uni‐modality features is ensured under the constraint of supervision. Finally, to make full use of the rich multi‐level and multi‐scale decoding features contained in TSD, the authors design the adaptive multi‐scale decoding module and the multi‐stream feature aggregation module to improve the decoding capability. Extensive experiments on three public datasets show that the MSTNet outperforms 14 state‐of‐the‐art methods, demonstrating that this method can extract and utilise the multi‐modality information more adequately and extract more complete and rich features, thus improving the model's performance. The code will be released at https://github.com/JOOOOKII/MSTNet.

Modality-Aware Adaptive-Integration Guided Single-Stream Network for RGB-T Saliency Detection

Multi-enhanced Adaptive Attention Network for RGB- T Salient Object Detection

Specificity-preserving RGB-D Saliency Detection

TSMSAN: A Three-Stream Multi-Scale Attentive Network for Video Saliency Detection.

CAE-Net: Cross-Modal Attention Enhancement Network for RGB-T Salient Object Detection

Interactive Context-Aware Network for RGB-T Salient Object Detection

Towards accurate RGB-D saliency detection with complementary attention and adaptive integration

Bidirectional Cross-Selective Attention Network for Video Salient Object Detection

Modality-Induced Transfer-Fusion Network for RGB-D and RGB-T Salient Object Detection

Transformer-based Network for RGB-D Saliency Detection

Leveraging modality‐specific and shared features for RGB‐T salient object detection

AMDFNet: Adaptive multi-level deformable fusion network for RGB-D saliency detection

RGB-T Salient Object Detection Via Fusing Multi-level CNN Features.

RGB-T salient object detection via CNN feature and result saliency map fusion

Modal Complementary Fusion Network for RGB-T Salient Object Detection

M 2rnet: Multi-modal and Multi-Scale Refined Network for RGB-D Salient Object Detection

Cross-Modality Double Bidirectional Interaction and Fusion Network for RGB-T Salient Object Detection

M2RNet: Multi-modal and Multi-scale Refined Network for RGB-D Salient Object Detection

Learning Multiscale Deep Features and SVM Regressors for Adaptive RGB-T Saliency Detection

Attentive Cross-Modal Fusion Network for RGB-D Saliency Detection

CAFCNet: Cross-modality asymmetric feature complement network for RGB-T salient object detection