Abstract:Most recent methods for RGB (red-green-blue)-thermal salient object detection (SOD) involve several floating-point operations and have numerous parameters, resulting in slow inference, especially on common processors, and impeding their deployment on mobile devices for practical applications. To address these problems, we propose a lightweight spatial boosting network (LSNet) for efficient RGB-thermal SOD with a lightweight MobileNetV2 backbone to replace a conventional backbone (e.g., VGG, ResNet). To improve feature extraction using a lightweight backbone, we propose a boundary boosting algorithm that optimizes the predicted saliency maps and reduces information collapse in low-dimensional features. The algorithm generates boundary maps based on predicted saliency maps without incurring additional calculations or complexity. As multimodality processing is essential for high-performance SOD, we adopt attentive feature distillation and selection and propose semantic and geometric transfer learning to enhance the backbone without increasing the complexity during testing. Experimental results demonstrate that the proposed LSNet achieves state-of-the-art performance compared with 14 RGB-thermal SOD methods on three datasets while improving the numbers of floating-point operations (1.025G) and parameters (5.39M), model size (22.1 MB), and inference speed (9.95 fps for PyTorch, batch size of 1, and Intel i5-7500 processor; 93.53 fps for PyTorch, batch size of 1, and NVIDIA TITAN V graphics processor; 936.68 fps for PyTorch, batch size of 20, and graphics processor; 538.01 fps for TensorRT and batch size of 1; and 903.01 fps for TensorRT/FP16 and batch size of 1). The code and results can be found from the link of https://github.com/zyrant/LSNet.

MoADNet: Mobile Asymmetric Dual-Stream Networks for Real-Time and Lightweight RGB-D Salient Object Detection

ADNet: An Asymmetric Dual-Stream Network for RGB-T Salient Object Detection.

MobileSal: Extremely Efficient RGB-D Salient Object Detection

A Single Stream Network for Robust and Real-Time RGB-D Salient Object Detection

TANet: Transformer-based Asymmetric Network for RGB-D Salient Object Detection

Cross-modal refined adjacent-guided network for RGB-D salient object detection

Depth Cue Enhancement and Guidance Network for RGB-D Salient Object Detection

SAMNet: Stereoscopically Attentive Multi-Scale Network for Lightweight Salient Object Detection

MMNet: Multi-Stage and Multi-Scale Fusion Network for RGB-D Salient Object Detection

Cross-Modal Fusion and Progressive Decoding Network for RGB-D Salient Object Detection

LSNet: Lightweight Spatial Boosting Network for Detecting Salient Objects in RGB-Thermal Images

Lightweight Cross-Modal Information Mutual Reinforcement Network for RGB-T Salient Object Detection

Lightweight Multi-modal Representation Learning for RGB Salient Object Detection

Real-Time One-Stream Semantic-Guided Refinement Network for RGB-Thermal Salient Object Detection

An adaptive guidance fusion network for RGB-D salient object detection

LARNet:Towards Lightweight, Accurate and Real-time Salient Object Detection

Cross-modality Discrepant Interaction Network for RGB-D Salient Object Detection

Unified Information Fusion Network for Multi-Modal RGB-D and RGB-T Salient Object Detection

Attention-Guided Multi-Modality Interaction Network for RGB-D Salient Object Detection

Modality-Guided Subnetwork for Salient Object Detection

Cross-Modal Weighting Network for RGB-D Salient Object Detection