Real-Time One-Stream Semantic-Guided Refinement Network for RGB-Thermal Salient Object Detection
Fushuo Huo,Xuegui Zhu,Qian Zhang,Ziming Liu,Wenchao Yu
DOI: https://doi.org/10.1109/tim.2022.3185323
IF: 5.6
2022-07-12
IEEE Transactions on Instrumentation and Measurement
Abstract:Salient object detection (SOD) has been widely used in practical applications, such as multisensor image fusion, remote sensing, and defect detection. Recently, SOD from RGB and thermal (T) has been rapidly developed due to its robustness to extreme situations, such as low illumination and occlusion. However, existing methods all utilize a dual-stream encoder, which significantly increases the computation burdens and hinders real-world deployment. To this end, we propose a real-time One-stream Semantic-guided Refinement Network (OSRNet) for RGB-T SOD. Specifically, we first fuse the RGB and T via concatenation, addition, and multiplication operations to dig the complementary information between each modality. The efficient early fusion not only facilitates the information exchange between each modality but also avoids the cumbersome dual-stream encoder structure. Then, the lightweight decoder is proposed, making the high-level semantic information filter the low-level noisy features and gradually refine the final prediction. Also, we apply deep supervision to make the training procedure more stable and fast. Due to the early fusion strategy, OSRNet can run at a real-time speed (53–60 fps) on a single GPU. Extensive quantitative and qualitative experiments show that our network outperforms 11 state-of-the-art methods in terms of seven evaluation metrics. Our codes have been released at https://github.com/huofushuo/OSRNet.
engineering, electrical & electronic,instruments & instrumentation