RASTMTrack: Robust and Adaptive Space-Time Memory Networks for Visual Tracking

Yu Zheng,Yong Liu,Xun Che

DOI: https://doi.org/10.1109/cyber63482.2024.10749627

2024-01-01

Abstract:These trackers based on the space-time memory network locate the target object in the search image employing contextual information from multiple memory frames and their corresponding foreground-background features. It is conceivable that these trackers are susceptible to the memory frame quality as well as the accuracy of the corresponding foreground labels. In the previous work (STMTrack), a fixed and experienced extraction strategy for memory frames was adopted, which hinders the improvement of performance and generalization. To address these limitations, we propose a dynamic and adaptive extraction strategy for memory frames to ensure that the most representative historical frames are selected into the set of memory frames to improve the accuracy of localization and reduce failures due to error accumulation. Specifically, we propose an extraction network to evaluate historical frames, where historical frames with the highest score are labeled as the memory frame and conversely dropped. Qualitative and quantitative analyses were implemented on multiple datasets (LaSOT and GOT-10K), and our method achieves significant performance gains (LaSOT: 60.6→61.4, GOT-10K: 64.2→65.2) over STMTrack especially for challenging scenarios (e.g., occlusion, vanishing, and reappearance, etc.). while bringing only a negligible inference speed degradation (Fps: 37 → 34), otherwise, the proposed method obtains competitive results compared to other state-of-the-art (SOTA) methods (CNN-based).