Abstract:The rapid development of deep learning provides a significant improvement of salient object detection combining both RGB and thermal images. However, existing deep learning-based models suffer from two major shortcomings. First, the computation and memory demands of Transformer-based models with quadratic complexity are unbearable, especially in handling high-resolution bi-modal feature fusion. Second, even if learning converges to an ideal solution, there remains a frequency gap between the prediction and ground truth. Therefore, we propose a purely fast Fourier transform-based model, namely deep Fourier-embedded network (DFENet), for learning bi-modal information of RGB and thermal images. On one hand, fast Fourier transform efficiently fetches global dependencies with low complexity. Inspired by this, we design modal-coordinated perception attention to fuse the frequency gap between RGB and thermal modalities with multi-dimensional representation enhancement. To obtain reliable detailed information during decoding, we design the frequency-decomposed edge-aware module (FEM) to clarify object edges by deeply decomposing low-level features. Moreover, we equip proposed Fourier residual channel attention block in each decoder layer to prioritize high-frequency information while aligning channel global relationships. On the other hand, we propose co-focus frequency loss (CFL) to steer FEM towards minimizing the frequency gap. CFL dynamically weights hard frequencies during edge frequency reconstruction by cross-referencing the bi-modal edge information in the Fourier domain. This frequency-level refinement of edge features further contributes to the quality of the final pixel-level prediction. Extensive experiments on four bi-modal salient object detection benchmark datasets demonstrate our proposed DFENet outperforms twelve existing state-of-the-art models.

Boosting Edge Detection Via Fusing Spatial and Frequency Domains

Edge Detection via Fusion Difference Convolution

ASFD: Automatic and Scalable Face Detector

FFEDet: Fine-Grained Feature Enhancement for Small Object Detection

FESSD:SSD target detection based on feature fusion and feature enhancement

Dynamic Feature Fusion for Semantic Edge Detection

Msmsfnet: a multi-stream and multi-scale fusion net for edge detection

SSD with multi-scale feature fusion and attention mechanism

FFR-SSD: feature fusion and reconstruction single shot detector for multi-scale object detection

Saliency Optimization Fused Background Feature with Frequency Domain Features

Edge Distraction-aware Salient Object Detection

One-Stage Deep Edge Detection Based on Dense-Scale Feature Fusion and Pixel-Level Imbalance Learning

Discriminative feature fusion for RGB-D salient object detection

Cross-scale information enhancement for object detection

Dual SIE-FPN: Semantic and Spatial Information Enhancement for Multiscale Object Detection

Enhancement-fusion Feature Pyramid Network for Object Detection

Deep Fourier-embedded Network for Bi-modal Salient Object Detection

Feature fusion method based on spiking neural convolutional network for edge detection

Spatial Frequency Enhanced Salient Object Detection

Single-Shot Object Detection via Feature Enhancement and Channel Attention

DeployFusion: A Deployable Monocular 3D Object Detection with Multi-Sensor Information Fusion in BEV for Edge Devices