Ship Detection in SAR Images with Human-in-the-Loop

Hecheng Jia,Feng Xu

2024-01-16

Abstract:Synthetic aperture radar (SAR) has been extensively utilized in maritime domains due to its all-weather, all-day monitoring capabilities, particularly exhibiting significant value in ship detection. In recent years, deep learning methods have increasingly been utilized for refined ship detection. However, learning-based methods exhibit poor generalization when confronted with new scenarios and data, necessitating expert intervention for continuous annotation. Currently, the degree of automation in human-machine collaboration within this field, especially in annotating new data, is not high, leading to labor- and computation-intensive model iteration and updates. Addressing these issues, a ship detection framework in SAR images with human-in-the-loop (HitL) is proposed. Incorporating the concept of HitL, tailored active learning strategies are designed for SAR ship detection tasks to present valuable samples to users, and an interactive human-machine interface (HMI) is established to efficiently collect user feedback. Consequently, user input is utilized in each interaction round to enhance model performance. Employing the proposed framework, an annotated ship database of SAR images is constructed, and the iteration experiments conducted during the construction demonstrates the efficiency of the method, providing new perspectives and approaches for research in this domain.

Image and Video Processing

What problem does this paper attempt to address?

The problems that this paper attempts to solve are as follows: In the ship detection task in Synthetic Aperture Radar (SAR) images, the existing deep - learning methods have poor generalization ability when facing new scenarios and data, and require continuous labeling by experts. In addition, currently, the degree of automation in human - machine collaboration is not high, especially in the labeling of new data, resulting in time - consuming and computationally - resource - intensive model iteration and updating. To solve these problems, the paper proposes a SAR ship - detection framework based on "Human - in - the - loop" (HitL).

Specifically, this paper aims to:
1. **Improve the model generalization ability**: By introducing the HitL framework, the model can gradually learn and adapt when encountering new data, reducing the dependence on large - scale labeled data.
2. **Enhance the automation level of human - machine collaboration**: Design specific active learning strategies, select valuable samples for user labeling, and establish an efficient interactive Human - Machine Interface (HMI) to quickly collect user feedback, thereby enhancing the model performance.
3. **Construct a high - quality SAR ship - detection database**: Through iterative experiments to verify the effectiveness of the proposed method, and finally form a database containing high - precision ship position labels and multiple fine - grained ship types.

### Main innovation points:
1. **Active learning strategy**: An active learning strategy suitable for the SAR ship - detection task is designed, which improves the efficiency of user labeling and sample correction, and helps users complete tasks more quickly.
2. **Interactive Human - Machine Interface (HMI)**: An efficient HMI is constructed, which uses human feedback to guide the iterative optimization of the model, making the model achieve the highest accuracy in the ship - detection task.
3. **SAR ship - detection database**: A high - quality SAR ship - detection database is constructed through the proposed cyclic framework, including finely - classified ship types and high - precision position labels.

### Formula explanation:
- **Softmax function** is used to normalize the probability distribution of classification results:
\[
S_j=\frac{e^{z_j}}{\sum_{k = 1}^K e^{z_k}}
\]
where \(S_j\) represents the probability score of the \(j\)-th category, \(z_j\) is the original logit value of the \(j\)-th category, and \(K\) is the total number of categories.

- **Uncertainty score** is used to evaluate the uncertainty of the classification task:
\[
U_r = 1-(S_{\text{top1}}-S_{\text{top2}})
\]
where \(U_r\) is the uncertainty score, and \(S_{\text{top1}}\) and \(S_{\text{top2}}\) are the highest and second - highest confidence scores among all categories in a single sample, respectively.

- **Uncertainty score for bounding box prediction**:
\[
C_i=\frac{1}{n - 1}\sum_{j = 1, j\neq i}^n\text{IoU}(B_i, B_j)
\]
\[
U_d=1-(C_{\text{top1}}-C_{\text{top2}})
\]
where \(C_i\) is the confidence score of the \(i\)-th bounding box, \(n\) is the number of inference times, \(B_i\) is the current bounding box, and \(U_d\) is the uncertainty of the current target prediction bounding box.

- **Comprehensive uncertainty score**:
\[
U_t=\frac{(1+\beta^2)\cdot U_r\cdot U_d}{\beta^2\cdot U_r + U_d}
\]
where \(\beta\) is a weighting factor, which is set to 1 here.

Through these methods, the paper effectively solves the human - machine collaboration and model generalization problems in SAR ship detection, providing new research perspectives and technical means.