Treatment of Irradiated Primates

D. V. Bekkum

DOI: https://doi.org/10.1126/science.138.3539.538

Abstract:

What problem does this paper attempt to address?

Salient Object Detection Based on Visual Perceptual Saturation and Two-Stream Hybrid Networks.

Chen Pan,Jianfeng Liu,Wei Qi Yan,Feilong Cao,Wei He,Yongxia Zhou

DOI: https://doi.org/10.1109/tip.2021.3074796

IF: 10.6

2021-01-01

IEEE Transactions on Image Processing

Abstract:Inspired by the perceived saturation of human visual system, this paper proposes a two-stream hybrid networks to simulate binocular vision for salient object detection (SOD). Each stream in our system consists of unsupervised and supervised methods to form a two-branch module, so as to model the interaction between human intuition and memory. The two-branch module parallel processes visual information with bottom-up and top-down SODs, and output two initial saliency maps. Then a polyharmonic neural network with random-weight (PNNRW) is utilized to fuse two-branch’s perception and refine the salient objects by learning online via multi-source cues. Depend on visual perceptual saturation, we can select optimal parameter of superpixel for unsupervised branch, locate sampling regions for PNNRW, and construct a positive feedback loop to facilitate perception saturated after the perception fusion. By comparing the binary outputs of the two-stream, the pixel annotation of predicted object with high saturation degree could be taken as new training samples. The presented method constitutes a semi-supervised learning framework actually. Supervised branches only need to be pre-trained initial, the system can collect the training samples with high confidence level and then train new models by itself. Extensive experiments show that the new framework can improve performance of the existing SOD methods, that exceeds the state-of-the-art methods in six popular benchmarks.
Dynamic Selective Network for RGB-D Salient Object Detection

Hongfa Wen,Chenggang Yan,Xiaofei Zhou,Runmin Cong,Yaoqi Sun,Bolun Zheng,Jiyong Zhang,Yongjun Bao,Guiguang Ding

DOI: https://doi.org/10.1109/TIP.2021.3123548

Abstract:RGB-D saliency detection is receiving more and more attention in recent years. There are many efforts have been devoted to this area, where most of them try to integrate the multi-modal information, i.e. RGB images and depth maps, via various fusion strategies. However, some of them ignore the inherent difference between the two modalities, which leads to the performance degradation when handling some challenging scenes. Therefore, in this paper, we propose a novel RGB-D saliency model, namely Dynamic Selective Network (DSNet), to perform salient object detection (SOD) in RGB-D images by taking full advantage of the complementarity between the two modalities. Specifically, we first deploy a cross-modal global context module (CGCM) to acquire the high-level semantic information, which can be used to roughly locate salient objects. Then, we design a dynamic selective module (DSM) to dynamically mine the cross-modal complementary information between RGB images and depth maps, and to further optimize the multi-level and multi-scale information by executing the gated and pooling based selection, respectively. Moreover, we conduct the boundary refinement to obtain high-quality saliency maps with clear boundary details. Extensive experiments on eight public RGB-D datasets show that the proposed DSNet achieves a competitive and excellent performance against the current 17 state-of-the-art RGB-D SOD models.
CSNet: a ConvNeXt-based Siamese network for RGB-D salient object detection

Yunhua Zhang,Hangxu Wang,Gang Yang,Jianhao Zhang,Congjin Gong,Yutao Wang

DOI: https://doi.org/10.1007/s00371-023-02887-x

2023-05-24

Abstract:Global contexts are critical to locating salient objects for salient object detection (SOD). However, the convolution operation in CNNs has a local receptive field, which cannot capture long-distance global information. Recent studies have shown that modernized CNN models with large kernel convolution, such as ConvNeXt, can effectively extend the receptive fields. Based on it, this paper explores the potential of large kernel CNN for SOD task. Inspired by the common information between RGB and depth images in salient objects, we propose a ConvNeXt-based Siamese network with shared weight parameters. This structural design can effectively reduce the number of parameters without sacrificing performance. Furthermore, a depth information preprocessing module is proposed to minimize the impact of low-quality depth images on predicted saliency maps. For cross-modal feature interaction, a dynamic fusion module is designed to enhance cross-modal complementarity dynamically. Extensive experiments and evaluation results on six benchmark datasets demonstrate the outstanding performance of the proposed method against 14 state-of-the-art RGB-D methods. Our code will be released at https://github.com/zyh5119232/CSNet.

computer science, software engineering
SPSN: Superpixel Prototype Sampling Network for RGB-D Salient Object Detection

Minhyeok Lee,Chaewon Park,Suhwan Cho,Sangyoun Lee

DOI: https://doi.org/10.48550/arXiv.2207.07898

2022-07-16

Computer Vision and Pattern Recognition

Abstract:RGB-D salient object detection (SOD) has been in the spotlight recently because it is an important preprocessing operation for various vision tasks. However, despite advances in deep learning-based methods, RGB-D SOD is still challenging due to the large domain gap between an RGB image and the depth map and low-quality depth maps. To solve this problem, we propose a novel superpixel prototype sampling network (SPSN) architecture. The proposed model splits the input RGB image and depth map into component superpixels to generate component prototypes. We design a prototype sampling network so that the network only samples prototypes corresponding to salient objects. In addition, we propose a reliance selection module to recognize the quality of each RGB and depth feature map and adaptively weight them in proportion to their reliability. The proposed method makes the model robust to inconsistencies between RGB images and depth maps and eliminates the influence of non-salient objects. Our method is evaluated on five popular datasets, achieving state-of-the-art performance. We prove the effectiveness of the proposed method through comparative experiments.
HFENet: Hybrid feature encoder network for detecting salient objects in RGB-thermal images

Fan Sun,Wujie Zhou,Weiqing Yan,Yulai Zhang

DOI: https://doi.org/10.1016/j.dsp.2024.104439

IF: 2.92

2024-05-01

Digital Signal Processing

Abstract:Deep convolutional neural networks (CNNs) have gained prominence in computer vision applications, including RGB salient object detection (SOD), owing to the advancements in deep learning. Nevertheless, the majority of deep CNNs employ either VGGNet or ResNet as their backbone architecture for extracting image information. This approach may lead to the following problems. 1) Variations between imaging modalities during feature extraction across layers. Cross-modal features across layers are often fused in a single step, resulting in inadequate cross-modal feature extraction. 2) Feature long-range dependence problem in multilayer feature decoding. 3) Image boundary blurring. To address these issues, we initially leverage the advantages offered by the VGGNet and ResNet architectures. Additionally, we present a novel hybrid VGG–ResNet feature encoder for RGB-T SOD. Specifically, we introduce a geometry information aggregation module that effectively combines and enhances the VGGNet spatial features of the RGB-T modalities from the bottom to the top. Moreover, we propose a innovative global saliency perception module that progressively refines the ResNet semantic features from the top to the bottom by integrating both local and global information. Furthermore, we introduce a Pearson-gated module to tackle the challenge of long-range dependence between features. This module utilizes gating to merge features by calculating the Pearson correlation coefficients of the fused features at multiple levels. Lastly, we devise an edge-aware module to precisely learn the contours of salient objects, thereby enhancing the clarity of the object boundaries. Extensive experiments conducted on three RGB-T SOD benchmarks demonstrate that our proposed network surpasses the performance of state-of-the-art methods for SOD.

engineering, electrical & electronic
Interactive Context-Aware Network for RGB-T Salient Object Detection

Yuxuan Wang,Feng Dong,Jinchao Zhu

DOI: https://doi.org/10.48550/arXiv.2211.06097

2022-11-11

Abstract:Salient object detection (SOD) focuses on distinguishing the most conspicuous objects in the scene. However, most related works are based on RGB images, which lose massive useful information. Accordingly, with the maturity of thermal technology, RGB-T (RGB-Thermal) multi-modality tasks attain more and more attention. Thermal infrared images carry important information which can be used to improve the accuracy of SOD prediction. To accomplish it, the methods to integrate multi-modal information and suppress noises are critical. In this paper, we propose a novel network called Interactive Context-Aware Network (ICANet). It contains three modules that can effectively perform the cross-modal and cross-scale fusions. We design a Hybrid Feature Fusion (HFF) module to integrate the features of two modalities, which utilizes two types of feature extraction. The Multi-Scale Attention Reinforcement (MSAR) and Upper Fusion (UF) blocks are responsible for the cross-scale fusion that converges different levels of features and generate the prediction maps. We also raise a novel Context-Aware Multi-Supervised Network (CAMSNet) to calculate the content loss between the prediction and the ground truth (GT). Experiments prove that our network performs favorably against the state-of-the-art RGB-T SOD methods.

Computer Vision and Pattern Recognition,Artificial Intelligence
MSEDNet: Multi-scale fusion and edge-supervised network for RGB-T salient object detection

Daogang Peng,Weiyi Zhou,Junzhen Pan,Danhao Wang

DOI: https://doi.org/10.1016/j.neunet.2023.12.031

Abstract:RGB-T Salient object detection (SOD) is to accurately segment salient regions in both visible light images and thermal infrared images. However, most of existing methods for SOD neglects the critical complementarity between multiple modalities images, which is beneficial to further improve the detection accuracy. Therefore, this work introduces the MSEDNet RGB-T SOD method. We utilize an encoder to extract multi-level modalities features from both visible light images and thermal infrared images, which are subsequently categorized into high, medium, and low level. Additionally, we propose three separate feature fusion modules to comprehensively extract complementary information between different modalities during the fusion process. These modules are applied to specific feature levels: the Edge Dilation Sharpening module for low-level features, the Spatial and Channel-Aware module for mid-level features, and the Cross-Residual Fusion module for high-level features. Finally, we introduce an edge fusion loss function for supervised learning, which effectively extracts edge information from different modalities and suppresses background noise. Comparative demonstrate the superiority of the proposed MSEDNet over other state-of-the-art methods. The code and results can be found at the following link: https://github.com/Zhou-wy/MSEDNet.
Cross-modal refined adjacent-guided network for RGB-D salient object detection

Hongbo Bi,Jiayuan Zhang,Ranwan Wu,Yuyu Tong,Wei Jin

DOI: https://doi.org/10.1007/s11042-023-14421-1

IF: 2.577

2023-03-22

Multimedia Tools and Applications

Abstract:RGB and depth modalities can be exploited to effectively recognize the most eye-catching objects in different scenes. Therefore, RGB-D salient object detection (RGB-D SOD) has been a popular direction focused by researchers. Particularly in recent years, various newfangled RGB-D SOD algorithms have been proposed endlessly and achieved outstanding performance. However, most approaches adopt the common pyramid structure to integrate multi-scale cues but ignore the complementarity of features in cross-layers. Besides, it is still challenging to fully utilize RGB and Depth information for cross-modal interaction. To compensate for these shortcomings, we propose a CRA-Net (Cross-modal Refined Adjacent-guided Network), which takes advantage of the high-level semantic information contained in the high layers to guide the details of the local characteristics in the low layers for improving detection accuracy. Specifically, a multiplier refinement module (MRM) is proposed to adequately carry out the information interaction between two modalities, in which a five-layer refinement mechanism is adopted to enhance cross-modal fusion representations. Moreover, for the purpose of obliterating the interference of non-significant factors in the low-level backgrounds, we design an adjacent-guided aggregation module (AAM). The multi-level features are fed in groups into two AAMs with identical structures. By utilizing an adjacent-layer guidance strategy to effectively guide multi-scale features assemblage from deep to shallow. Numerous experiments show that our CRA-Net is competitive for four common evaluation metrics on four popular datasets.

computer science, information systems, theory & methods,engineering, electrical & electronic, software engineering
RGB-D Salient Object Detection with Cross-Modality Modulation and Selection

Chongyi Li,Runmin Cong,Yongri Piao,Qianqian Xu,Chen Change Loy

DOI: https://doi.org/10.48550/arXiv.2007.07051

2020-07-14

Abstract:We present an effective method to progressively integrate and refine the cross-modality complementarities for RGB-D salient object detection (SOD). The proposed network mainly solves two challenging issues: 1) how to effectively integrate the complementary information from RGB image and its corresponding depth map, and 2) how to adaptively select more saliency-related features. First, we propose a cross-modality feature modulation (cmFM) module to enhance feature representations by taking the depth features as prior, which models the complementary relations of RGB-D data. Second, we propose an adaptive feature selection (AFS) module to select saliency-related features and suppress the inferior ones. The AFS module exploits multi-modality spatial feature fusion with the self-modality and cross-modality interdependencies of channel features are considered. Third, we employ a saliency-guided position-edge attention (sg-PEA) module to encourage our network to focus more on saliency-related regions. The above modules as a whole, called cmMS block, facilitates the refinement of saliency features in a coarse-to-fine fashion. Coupled with a bottom-up inference, the refined saliency features enable accurate and edge-preserving SOD. Extensive experiments demonstrate that our network outperforms state-of-the-art saliency detectors on six popular RGB-D SOD benchmarks.

Computer Vision and Pattern Recognition
M$^3$Net: Multilevel, Mixed and Multistage Attention Network for Salient Object Detection

Yao Yuan,Pan Gao,XiaoYang Tan

DOI: https://doi.org/10.48550/arXiv.2309.08365

2023-09-15

Abstract:Most existing salient object detection methods mostly use U-Net or feature pyramid structure, which simply aggregates feature maps of different scales, ignoring the uniqueness and interdependence of them and their respective contributions to the final prediction. To overcome these, we propose the M$^3$Net, i.e., the Multilevel, Mixed and Multistage attention network for Salient Object Detection (SOD). Firstly, we propose Multiscale Interaction Block which innovatively introduces the cross-attention approach to achieve the interaction between multilevel features, allowing high-level features to guide low-level feature learning and thus enhancing salient regions. Secondly, considering the fact that previous Transformer based SOD methods locate salient regions only using global self-attention while inevitably overlooking the details of complex objects, we propose the Mixed Attention Block. This block combines global self-attention and window self-attention, aiming at modeling context at both global and local levels to further improve the accuracy of the prediction map. Finally, we proposed a multilevel supervision strategy to optimize the aggregated feature stage-by-stage. Experiments on six challenging datasets demonstrate that the proposed M$^3$Net surpasses recent CNN and Transformer-based SOD arts in terms of four metrics. Codes are available at <a class="link-external link-https" href="https://github.com/I2-Multimedia-Lab/M3Net" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition,Artificial Intelligence
Compensated Attention Feature Fusion and Hierarchical Multiplication Decoder Network for RGB-D Salient Object Detection

Zhihong Zeng,Haijun Liu,Fenglei Chen,Xiaoheng Tan

DOI: https://doi.org/10.3390/rs15092393

IF: 5

2023-05-04

Remote Sensing

Abstract:Multi-modal feature fusion and effectively exploiting high-level semantic information are critical in salient object detection (SOD). However, the depth maps complementing RGB image fusion strategies cannot supply effective semantic information when the object is not salient in the depth maps. Furthermore, most existing (UNet-based) methods cannot fully exploit high-level abstract features to guide low-level features in a coarse-to-fine fashion. In this paper, we propose a compensated attention feature fusion and hierarchical multiplication decoder network (CAF-HMNet) for RGB-D SOD. Specifically, we first propose a compensated attention feature fusion module to fuse multi-modal features based on the complementarity between depth and RGB features. Then, we propose a hierarchical multiplication decoder to refine the multi-level features from top down. Additionally, a contour-aware module is applied to enhance object contour. Experimental results show that our model achieves satisfactory performance on five challenging SOD datasets, including NJU2K, NLPR, STERE, DES, and SIP, which verifies the effectiveness of the proposed CAF-HMNet.

environmental sciences,imaging science & photographic technology,remote sensing,geosciences, multidisciplinary
SLMSF-Net: A Semantic Localization and Multi-Scale Fusion Network for RGB-D Salient Object Detection

Yanbin Peng,Zhinian Zhai,Mingkun Feng

DOI: https://doi.org/10.3390/s24041117

IF: 3.9

2024-02-09

Sensors

Abstract:Salient Object Detection (SOD) in RGB-D images plays a crucial role in the field of computer vision, with its central aim being to identify and segment the most visually striking objects within a scene. However, optimizing the fusion of multi-modal and multi-scale features to enhance detection performance remains a challenge. To address this issue, we propose a network model based on semantic localization and multi-scale fusion (SLMSF-Net), specifically designed for RGB-D SOD. Firstly, we designed a Deep Attention Module (DAM), which extracts valuable depth feature information from both channel and spatial perspectives and efficiently merges it with RGB features. Subsequently, a Semantic Localization Module (SLM) is introduced to enhance the top-level modality fusion features, enabling the precise localization of salient objects. Finally, a Multi-Scale Fusion Module (MSF) is employed to perform inverse decoding on the modality fusion features, thus restoring the detailed information of the objects and generating high-precision saliency maps. Our approach has been validated across six RGB-D salient object detection datasets. The experimental results indicate an improvement of 0.20~1.80%, 0.09~1.46%, 0.19~1.05%, and 0.0002~0.0062, respectively in maxF, maxE, S, and MAE metrics, compared to the best competing methods (AFNet, DCMF, and C2DFNet).

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation
Cross-Modal Fusion and Progressive Decoding Network for RGB-D Salient Object Detection

Xihang Hu,Fuming Sun,Jing Sun,Fasheng Wang,Haojie Li

DOI: https://doi.org/10.1007/s11263-024-02020-y

IF: 13.369

2024-03-03

International Journal of Computer Vision

Abstract:Most existing RGB-D salient object detection (SOD) methods tend to achieve higher performance by integrating additional modules, such as feature enhancement and edge generation. There is no doubt that these modules will inevitably produce feature redundancy and performance degradation. To this end, we exquisitely design a cross-modal fusion and progressive decoding network (termed CPNet) to achieve RGB-D SOD tasks. The designed network structure only includes three indispensable parts: feature encoding, feature fusion and feature decoding. Specifically, in the feature encoding part, we adopt a two-stream Swin Transformer encoder to extract multi-level and multi-scale features from RGB images and depth images respectively to model global information. In the feature fusion part, we design a cross-modal attention fusion module, which can leverage the attention mechanism to fuse multi-modality and multi-level features. In the feature decoding part, we design a progressive decoder to gradually fuse low-level features and filter noise information to accurately predict salient objects. Extensive experimental results on 6 benchmarks demonstrated that our network surpasses 12 state-of-the-art methods in terms of four metrics. In addition, it is also verified that for the RGB-D SOD task, the addition of the feature enhancement module and the edge generation module is not conducive to improving the detection performance under this framework, which provides new insights into the salient object detection task. Our codes are available at https://github.com/hu-xh/CPNet.

computer science, artificial intelligence
Global-prior-guided fusion network for salient object detection

Peng Peng,Kai-Fu Yang,Yong-Jie Li

DOI: https://doi.org/10.1016/j.eswa.2022.116805

IF: 8.5

2022-07-01

Expert Systems with Applications

Abstract:RGB image and depth map (RGB-D)-based salient object detection (SOD) has been well-studied in recent years, especially using deep neural networks. An RGB image provides rich local and semantic features, while the depth map provides global structural information. Many researchers have treated depth information as a supplement to RGB maps. However, depth maps in various datasets are not as precise as RGB information, as they are captured under various conditions. Therefore, thoroughly exploiting these features at different levels remains unresolved. Many cognitive theories, such as the topological perception theory, claim that global properties are prior to local ones and are important for human recognition. In this paper, we propose a novel global-prior-guided fusion network with global-prior extraction modules to fuse cross-modality features. Each module contains a cross attention guided by deeper global priors, and the global prior extracted by this module is used to guide the processing of local features in shallow layers. The global guided network first integrates the local and global cross features into the decoder of depth maps, and then the fused structural features of the decoder are finally fused into the saliency decoder. Experimental results show that our method outperformed other state-of-the-art methods in the RGB-D-based SOD task on seven datasets (i.e., DUT-RGBD, NJUD, LFSD, NLPR, RGBD135, SIP, and STERE) and in terms of most metrics. To thoroughly exploit the modules we designed, we extended our model to accomplish the tasks of RGB and video SOD with slight adaptions, and obtained results comparable to those of the state-of-the-art (SOTA) methods in both fields.

computer science, artificial intelligence,engineering, electrical & electronic,operations research & management science
M2RNet: Multi-modal and Multi-scale Refined Network for RGB-D Salient Object Detection

Xian Fang,Jinchao Zhu,Ruixun Zhang,Xiuli Shao,Hongpeng Wang

DOI: https://doi.org/10.48550/arXiv.2109.07922

2021-09-16

Abstract:Salient object detection is a fundamental topic in computer vision. Previous methods based on RGB-D often suffer from the incompatibility of multi-modal feature fusion and the insufficiency of multi-scale feature aggregation. To tackle these two dilemmas, we propose a novel multi-modal and multi-scale refined network (M2RNet). Three essential components are presented in this network. The nested dual attention module (NDAM) explicitly exploits the combined features of RGB and depth flows. The adjacent interactive aggregation module (AIAM) gradually integrates the neighbor features of high, middle and low levels. The joint hybrid optimization loss (JHOL) makes the predictions have a prominent outline. Extensive experiments demonstrate that our method outperforms other state-of-the-art approaches.

Computer Vision and Pattern Recognition
Multi-level cross-modal interaction network for RGB-D salient object detection

Zhou Huang,Huai-Xin Chen,Tao Zhou,Yun-Zhi Yang,Bi-Yuan Liu

DOI: https://doi.org/10.1016/j.neucom.2021.04.053

IF: 6

2021-09-01

Neurocomputing

Abstract:Depth cues with affluent spatial information have been proven beneficial in boosting salient object detection (SOD), while the depth quality directly affects the subsequent SOD performance. However, it is inevitable to obtain some low-quality depth cues due to limitations of its acquisition devices, which can inhibit the SOD performance. Besides, existing methods tend to combine RGB images and depth cues in a direct fusion or a simple fusion module, which makes they can not effectively exploit the complex correlations between the two sources. Moreover, few methods design an appropriate module to fully fuse multi-level features, resulting in cross-level feature interaction insufficient. To address these issues, we propose a novel Multi-level Cross-modal Interaction Network (MCI-Net) for RGB-D based SOD. Our MCI-Net includes two key components: 1) a cross-modal feature learning network, which is used to learn the high-level features for the RGB images and depth cues, effectively enabling the correlations between the two sources to be exploited; and 2) a multi-level interactive integration network, which integrates multi-level cross-modal features to boost the SOD performance. Extensive experiments on six benchmark datasets demonstrate the superiority of our MCI-Net over 14 state-of-the-art methods, and validate the effectiveness of different components in our MCI-Net . More important, our MCI-Net significantly improves the SOD performance as well as has a higher FPS.

computer science, artificial intelligence
Attention-guided cross-modal multiple feature aggregation network for RGB-D salient object detection

Bojian Chen,Wenbin Wu,Zhezhou Li,Tengfei Han,Zhuolei Chen,Weihao Zhang

DOI: https://doi.org/10.3934/era.2024031

2024-01-01

Electronic Research Archive

Abstract:The goal of RGB-D salient object detection is to aggregate the information of the two modalities of RGB and depth to accurately detect and segment salient objects. Existing RGB-D SOD models can extract the multilevel features of single modality well and can also integrate cross-modal features, but it can rarely handle both at the same time. To tap into and make the most of the correlations of intra- and inter-modality information, in this paper, we proposed an attention-guided cross-modal multi-feature aggregation network for RGB-D SOD. Our motivation was that both cross-modal feature fusion and multilevel feature fusion are crucial for RGB-D SOD task. The main innovation of this work lies in two points: One is the cross-modal pyramid feature interaction (CPFI) module that integrates multilevel features from both RGB and depth modalities in a bottom-up manner, and the other is cross-modal feature decoder (CMFD) that aggregates the fused features to generate the final saliency map. Extensive experiments on six benchmark datasets showed that the proposed attention-guided cross-modal multiple feature aggregation network (ACFPA-Net) achieved competitive performance over 15 state of the art (SOTA) RGB-D SOD methods, both qualitatively and quantitatively.

mathematics
DeepSaliency : MultiTask Deep Neural Network Model for Salient Object Detection

Xi Li,Liming Zhao,Lina Wei,Ming-Hsuan Yang,Fei Wu,Yueting Zhuang,Haibin Ling,Jingdong Wang

DOI: https://doi.org/10.1109/tip.2016.2579306

IF: 10.6

2016-01-01

IEEE Transactions on Image Processing

Abstract:A key problem in salient object detection is how to effectively model the semantic properties of salient objects in a data-driven manner. In this paper, we propose a multi-task deep saliency model based on a fully convolutional neural network with global input (whole raw images) and global output (whole saliency maps). In principle, the proposed saliency model takes a data-driven strategy for encoding the underlying saliency prior information, and then sets up a multi-task learning scheme for exploring the intrinsic correlations between saliency detection and semantic image segmentation. Through collaborative feature learning from such two correlated tasks, the shared fully convolutional layers produce effective features for object perception. Moreover, it is capable of capturing the semantic information on salient objects across different levels using the fully convolutional layers, which investigate the feature-sharing properties of salient object detection with a great reduction of feature redundancy. Finally, we present a graph Laplacian regularized nonlinear regression model for saliency refinement. Experimental results demonstrate the effectiveness of our approach in comparison with the state-of-the-art approaches.
Bi-directional Progressive Guidance Network for RGB-D Salient Object Detection

Yang Yang,Qi Qin,Yongjiang Luo,Yi Liu,Qiang Zhang,Jungong Han

DOI: https://doi.org/10.1109/tcsvt.2022.3144852

IF: 5.859

2022-01-01

IEEE Transactions on Circuits and Systems for Video Technology

Abstract:Most existing RGB-D salient detection models pay more attention to the quality of the depth images, while in some special cases, the quality of RGB images may even have greater impacts on saliency detection, which has long been ignored and underestimated. To address this problem, in this paper, we present a Bi-directional Progressive Guidance Network (BPGNet) for RGB-D salient object detection, where the qualities of both RGB and depth images are involved. Since it is usually difficult to determine which modality data have low quality in advance, a bi-directional framework based on progressive guidance (PG) strategy is employed to extract and enhance the unimodal features with the aid of another modality data via the alternative interactions between the saliency prediction results and the extracted features from the multi-modality input data. Specifically, the proposed PG strategy is achieved by using the proposed Global Context Awareness (GCA), Auxiliary Feature Extraction (AFE) and Cross-modality Feature Enhancement (CFE) modules. Benefiting from the proposed PG strategy, the disturbing information within the input RGB and depth images can be well suppressed, while the discriminative information within the input images gets enhanced. On top of that, a Fusion Prediction Module (FPM) is further designed to adaptively select those features with higher discriminability as well as enhancing the common information for the final saliency prediction. Experimental results demonstrate that our proposed model is comparable to those of state-of-the-art RGB-D SOD models.

engineering, electrical & electronic
Feature Calibrating and Fusing Network for RGB-D Salient Object Detection

Qiang Zhang,Qi Qin,Yang Yang,Qiang Jiao,Jungong Han

DOI: https://doi.org/10.1109/tcsvt.2023.3296581

IF: 5.859

2023-01-01

IEEE Transactions on Circuits and Systems for Video Technology

Abstract:Due to their imaging mechanisms and techniques, some depth images inevitably have low visual qualities or have some inconsistent foregrounds with their corresponding RGB images. Directly using such depth images will deteriorate the performance of RGB-D SOD. In view of this, a novel RGB-D salient object detection model is presented, which follows the principle of calibration-then-fusion to effectively suppress the influence of such two types of depth images on final saliency prediction. Specifically, the proposed model is composed of two stages, i.e., an image generation stage and a saliency reasoning stage. The former generates high-quality and foreground-consistent pseudo depth images via an image generation network. While the latter first calibrates the original depth information with the aid of those newly generated pseudo depth images and then performs cross-modal feature fusion for the final saliency reasoning. Especially, in the first stage, a Two-steps Sample Selection (TSS) strategy is employed to select such reliable depth images from the original RGB-D image pairs as supervision information to optimize the image generation network. Afterwards, in the second stage, a Feature Calibrating and Fusing Network (FCFNet) is proposed to achieve the calibration-then-fusion of cross-modal information for the final saliency prediction, which is achieved by a Depth Feature Calibration (DFC) module, a Shallow-level Feature Injection (SFI) module and a Multi-modal Multi-scale Fusion (MMF) module. Moreover, a loss function, i.e., Region Consistency Aware (RCA) loss, is presented as an auxiliary loss for FCFNet to facilitate the completeness of salient objects together with the reduction of background interference by considering the local regional consistency in the saliency maps. Experiments on six benchmark datasets demonstrate the superiorities of our proposed RGB-D SOD model over some state-of-the-arts.

engineering, electrical & electronic

Treatment of Irradiated Primates

Salient Object Detection Based on Visual Perceptual Saturation and Two-Stream Hybrid Networks.

Dynamic Selective Network for RGB-D Salient Object Detection

CSNet: a ConvNeXt-based Siamese network for RGB-D salient object detection

SPSN: Superpixel Prototype Sampling Network for RGB-D Salient Object Detection

HFENet: Hybrid feature encoder network for detecting salient objects in RGB-thermal images

Interactive Context-Aware Network for RGB-T Salient Object Detection

MSEDNet: Multi-scale fusion and edge-supervised network for RGB-T salient object detection

Cross-modal refined adjacent-guided network for RGB-D salient object detection

RGB-D Salient Object Detection with Cross-Modality Modulation and Selection

M$^3$Net: Multilevel, Mixed and Multistage Attention Network for Salient Object Detection

Compensated Attention Feature Fusion and Hierarchical Multiplication Decoder Network for RGB-D Salient Object Detection

SLMSF-Net: A Semantic Localization and Multi-Scale Fusion Network for RGB-D Salient Object Detection

Cross-Modal Fusion and Progressive Decoding Network for RGB-D Salient Object Detection

Global-prior-guided fusion network for salient object detection

M2RNet: Multi-modal and Multi-scale Refined Network for RGB-D Salient Object Detection

Multi-level cross-modal interaction network for RGB-D salient object detection

Attention-guided cross-modal multiple feature aggregation network for RGB-D salient object detection

DeepSaliency : MultiTask Deep Neural Network Model for Salient Object Detection

Bi-directional Progressive Guidance Network for RGB-D Salient Object Detection

Feature Calibrating and Fusing Network for RGB-D Salient Object Detection