Sea You Later: Metadata-Guided Long-Term Re-Identification for UAV-Based Multi-Object Tracking

Cheng-Yen Yang,Hsiang-Wei Huang,Zhongyu Jiang,Heng-Cheng Kuo,Jie Mei,Chung-I Huang,Jenq-Neng Hwang

2023-11-23

Abstract:Re-identification (ReID) in multi-object tracking (MOT) for UAVs in maritime computer vision has been challenging for several reasons. More specifically, short-term re-identification (ReID) is difficult due to the nature of the characteristics of small targets and the sudden movement of the drone's gimbal. Long-term ReID suffers from the lack of useful appearance diversity. In response to these challenges, we present an adaptable motion-based MOT algorithm, called Metadata Guided MOT (MG-MOT). This algorithm effectively merges short-term tracking data into coherent long-term tracks, harnessing crucial metadata from UAVs, including GPS position, drone altitude, and camera orientations. Extensive experiments are conducted to validate the efficacy of our MOT algorithm. Utilizing the challenging SeaDroneSee tracking dataset, which encompasses the aforementioned scenarios, we achieve a much-improved performance in the latest edition of the UAV-based Maritime Object Tracking Challenge with a state-of-the-art HOTA of 69.5% and an IDF1 of 85.9% on the testing split.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper primarily addresses the issues of multi-object tracking (MOT) and re-identification (ReID) of unmanned aerial vehicles (UAVs) in maritime environments. Specifically, the paper tackles the following key problems:

1. **Challenges of Multi-Object Tracking in Maritime Environments**:
- Small Target Detection: Maritime targets (such as ships, swimmers, etc.) are relatively small and their sizes change with the altitude of the UAV.
- Visual Condition Challenges: Factors like wave reflections lead to complex visual conditions.
- Dynamic Targets: Objects move with waves or changes in UAV posture.
- Partial Occlusion: Targets in maritime scenes are often partially occluded.

2. **Difficulties in Short-term and Long-term Re-identification**:
- Short-term ReID: Difficulty in tracking targets due to the rapid movement of UAVs or cameras.
- Long-term ReID: Accurate identification of targets that temporarily disappear and reappear, especially when targets have similar appearances (e.g., ships).

To address the above issues, the authors propose a "Metadata Guided Multi-Object Tracking Algorithm" (MG-MOT), which leverages the rich metadata provided by UAVs (such as GPS location, UAV altitude, camera orientation, etc.) to improve tracking and re-identification accuracy. The paper particularly emphasizes how to combine this metadata to construct a camera model and use this model for 3D geometric short-term and long-term re-identification.

Experimental results show that the proposed MG-MOT method achieves significant performance improvements on the SeaDroneSee-MOT dataset, obtaining the best HOTA (69.5%) and IDF1 (85.9%) scores in the latest UAV maritime multi-object tracking challenge. This demonstrates that the method can effectively address the challenges of multi-object tracking and re-identification in maritime environments.