Cross-modal Co-Feedback Cellular Automata for RGB-T Saliency Detection

Yu Pang,Hao Wu,Chengdong Wu
DOI: https://doi.org/10.1016/j.patcog.2022.109138
IF: 8
2022-01-01
Pattern Recognition
Abstract:Saliency cellular automata (CA), a temporally evolving model to efficiently locate salient object, has achieved great progress in saliency detection task. However, most the previous CA models originate from RGB data and are thus limited in some extreme scenes. Inspired by the observation that thermal in-frared data (T) can overcome the limitation of RGB data themselves in some cases and RGB-T saliency detection has gained more and more attention. In this paper, we contribute a novel RGB-T saliency de-tection approach via Cross-modal Co-feedback Cellular Automata (C3A). Before this, we firstly present a novel weighted background-based map (WBM) to give each superpixel(image patch) an initial saliency value. Then C3A is proposed to improve the quality of the WBM. Specifically, it firstly establishes two complementary cellular automata (CA) mechanisms dependent on RGB and thermal infrared data, which respectively refine the WBM based on two different perspectives. To bridge RGB-T modalities, an iterative cross-modal co-feedback framework is contributed to optimize constantly their results. In other words, we regard the result of one modality(RGB or T)-induced CA as important feedback to update and opti-mize another modality(T or RGB)-induced CA during the iteration. Two modalities constantly pull out the useful and confident data to the opposite side, and so two CAs' results are constantly refined until a sta-ble state is generated, we then integrate the results of two modalities-induced CAs into the saliency map. Finally, a novel boundary-guided pixel-wise refinement (BPR) technology is proposed to further overcome the influence of inaccurate superpixel segmentation to the C3A and refine the saliency map generated by our C3A. For fairness, the proposed method is compared with other state-of-the-art methods on three RGB-T datasets, experimental results show the superiority of our model.(c) 2022 Elsevier Ltd. All rights reserved.
What problem does this paper attempt to address?