Yasod Ginige,Ransika Gunasekara,Darsha Hewavitharana,Manjula Ariyarathne,Ranga Rodrigo,Peshala Jayasekara
Abstract:Maritime surveillance is vital to mitigate illegal activities such as drug smuggling, illegal fishing, and human trafficking. Vision-based maritime surveillance is challenging mainly due to visibility issues at night, which results in failures in re-identifying vessels and detecting suspicious activities. In this paper, we introduce a thermal, vision-based approach for maritime surveillance with object tracking, vessel re-identification, and suspicious activity detection capabilities. For vessel re-identification, we propose a novel viewpoint-independent algorithm which compares features of the sides of the vessel separately (separate side-spaces) leveraging shape information in the absence of color features. We propose techniques to adapt tracking and activity detection algorithms for the thermal domain and train them using a thermal dataset we created. This dataset will be the first publicly available benchmark dataset for thermal maritime surveillance. Our system is capable of re-identifying vessels with an 81.8% Top1 score and identifying suspicious activities with a 72.4\% frame mAP score; a new benchmark for each task in the thermal domain.
What problem does this paper attempt to address?
### What problems does this paper attempt to solve?
This paper aims to solve some key challenges in maritime surveillance, especially the difficulties faced by vision - based maritime surveillance systems in re - identifying ships and detecting suspicious activities at night and under extreme weather conditions. Specifically, the paper mainly focuses on the following issues:
1. **Visibility problems at night and under extreme weather conditions**:
- Maritime surveillance at night is difficult for traditional RGB - image - based surveillance methods to work effectively due to insufficient light, water surface reflection and the influence of bad weather conditions (such as fog, rain, etc.).
2. **Challenges in ship re - identification**:
- In the field of thermal imaging, the features of ships are different from those in RGB images, lacking color and detail features. Therefore, traditional RGB - based ship re - identification algorithms perform poorly on thermal imaging data.
- Ships have great differences in appearance from different viewing angles, making re - identification more difficult.
3. **Problems in suspicious activity detection**:
- Maritime illegal activities (such as smuggling, illegal fishing, human trafficking, etc.) need to be detected and responded to in a timely manner. However, existing RGB - image - based activity detection algorithms cannot be directly applied to thermal imaging data.
### Main contributions of the paper
To address the above challenges, the paper proposes the following solutions:
1. **Proposes a novel view - independent ship re - identification algorithm**:
- This algorithm utilizes shape information in thermal imaging instead of relying on color or detail features. It achieves re - identification by comparing feature vectors of different sides of ships, thus overcoming the problem of lack of color and detail features in thermal imaging data.
- Experimental results show that the mAP score of this algorithm in the thermal imaging domain is 32% higher than that of the SPAN model.
2. **Creates a public thermal - imaging - based maritime surveillance data set**:
- This data set contains annotated video clips, covering various maritime objects (such as ships, motorboats, human activities, etc.), and provides annotations in COCO format. This is the first public thermal - imaging - based maritime surveillance data set, which can be used for detection, tracking, re - identification and activity detection tasks.
3. **An adaptation of detection and tracking algorithms in the thermal - imaging field**:
- Researchers adjusted the TraDes algorithm originally trained in the RGB domain into a multi - target tracking algorithm suitable for the thermal - imaging field and fine - tuned it on thermal - imaging data. Experimental results show that the MOTA score of this algorithm in the thermal - imaging domain is 61.2%.
4. **An adaptation of activity detection algorithms in the thermal - imaging field**:
- Researchers extended the YOWO algorithm from the RGB domain to the thermal - imaging domain to detect suspicious activities such as possible human trafficking and swimming. Through retraining and adjusting hyper - parameters, the frame mAP score of this algorithm in the thermal - imaging domain is 62.45%, showing its effective detection ability for target activities.
### Summary
In general, this paper solves the limitations of traditional RGB images in maritime surveillance at night and under extreme weather conditions by introducing thermal - imaging technology, and provides a complete set of solutions, including ship re - identification, multi - target tracking and suspicious activity detection, significantly improving the performance and reliability of the maritime surveillance system.