Beyond Full Labels: Energy-Double-Guided Single-Point Prompt for Infrared Small Target Label Generation

Shuai Yuan,Hanlin Qin,Renke Kou,Xiang Yan,Zechuan Li,Chenxu Peng,Huixin Zhou
2024-11-16
Abstract:We pioneer a learning-based single-point prompt paradigm for infrared small target label generation (IRSTLG) to lobber annotation burdens. Unlike previous clustering-based methods, our intuition is that point-guided mask generation just requires one more prompt than target detection, i.e., IRSTLG can be treated as an infrared small target detection (IRSTD) with the location hint. Therefore, we propose an elegant yet effective Energy-Double-Guided Single-point Prompt (EDGSP) framework, aiming to adeptly transform a coarse IRSTD network into a refined label generation method. Specifically, EDGSP comprises three key modules: 1) target energy initialization (TEI), which establishes a foundational outline to streamline the mapping process for effective shape evolution, 2) double prompt embedding (DPE) for rapidly localizing interesting regions and reinforcing high-resolution individual edges to avoid label adhesion, and 3) bounding box-based matching (BBM) for eliminating false masks via considering comprehensive cluster boundary conditions to obtain a reliable output. In this way, pseudo labels generated by three backbones equipped with our EDGSP achieve 100% object-level probability of detection (Pd) and 0% false-alarm rate (Fa) on SIRST, NUDT-SIRST, and IRSTD-1k datasets, with a pixel-level intersection over union (IoU) improvement of 13.28% over state-of-the-art (SOTA) label generation methods. Further applying our inferred masks to train detection models, EDGSP, for the first time, enables a single-point-generated pseudo mask to surpass the manual labels. Even with coarse single-point annotations, it still achieves 99.5% performance of full labeling. Code is available at <a class="link-external link-https" href="https://github.com/xdFai/EDGSP" rel="external noopener nofollow">this https URL</a>.
Computer Vision and Pattern Recognition
What problem does this paper attempt to address?
### What problem does this paper attempt to solve? This paper aims to solve the problem that manual pixel - level labeling in infrared small target detection (IRSTD) is time - consuming and error - prone. Specifically, the traditional manual pixel - by - pixel labeling method faces many challenges when dealing with infrared small targets, including limited pixels, low signal - to - noise ratio, and unclear target edges. These problems make the labeling process both time - consuming and error - prone. To address these challenges, the authors propose a learning - based framework with single - point prompts - **Energy - Double - Guided Single - point Prompt (EDGSP)** - for generating pseudo - labels of infrared small targets. By introducing this framework, the authors hope to reduce the labeling burden and improve the quality of pseudo - labels, enabling them to surpass traditional manually labeled labels and thus better train the detection model. ### Main contributions 1. **Propose a learning - based infrared small target labeling paradigm for the first time**: Transform the existing infrared small target detection network into an advanced label generation model. 2. **Introduce three key modules**: - **Target Energy Initialization (TEI)**: Simplify the shape evolution process and provide a more sufficient representation of the target contour. - **Double Prompt Embedding (DPE)**: Enhance the discrimination of adjacent targets and prevent label adhesion. - **Bounding Box - based Matching (BBM)**: Eliminate incorrect masks by comprehensively considering clustering boundary conditions to obtain reliable output. 3. **Significantly improve the quality of pseudo - labels**: On three public datasets, the pseudo - labels generated by EDGSP outperform existing methods in multiple metrics, and their performance in the detection task even exceeds that of fully labeled data. ### Presentation of formulas in Markdown format - **Formulas in Target Energy Initialization (TEI)**: \[ I_p(x, y) = \begin{cases} 1 & \text{if } (x, y) = (x_0, y_0) \\ 0 & \text{otherwise} \end{cases} \] \[ G(x, y | x_0, y_0) = \exp\left(-\frac{(x - x_0)^2 + (y - y_0)^2}{2\sigma^2}\right) \] \[ I_g(x, y) = \sum_{(x_0, y_0) \in S} G(x, y | x_0, y_0) \] - **Formulas in Double Prompt Embedding (DPE)**: \[ I_t = \text{Concat}(I_i, S(I_i), I_g) \] \[ L = G_{\text{Conv}}^{\text{w/o}}(I_t) \] \[ O_s = \text{Sigmoid}(\text{Conv}(\text{Concat}(L, I_g))) \] - **Formulas in Bounding Box - based Matching (BBM)**: \[ O_c = A_{m \times n} \odot (O_s > \tau_S) \] \[ F_{\text{at}} = \frac{T_{\text{false}}}{N_{\text{all}}} \] Through these improvements, the EDGSP framework not only reduces the labeling burden but also significantly improves the quality of pseudo - labels, making it more practical in real - world applications.