Abstract:Visual tracking performance has long been limited by the lack of better appearance models. These models fail either where they tend to change rapidly, like in motion-based tracking, or where accurate information of the object may not be available, like in color camouflage (where background and foreground colors are similar). This paper proposes a robust, adaptive appearance model which works accurately in situations of color camouflage, even in the presence of complex natural objects. The proposed model includes depth as an additional feature in a hierarchical modular neural framework for online object tracking. The model adapts to the confusing appearance by identifying the stable property of depth between the target and the surrounding object(s). The depth complements the existing RGB features in scenarios when RGB features fail to adapt, hence becoming unstable over a long duration of time. The parameters of the model are learned efficiently in the Deep network, which consists of three modules: (1) The spatial attention layer, which discards the majority of the background by selecting a region containing the object of interest; (2) the appearance attention layer, which extracts appearance and spatial information about the tracked object; and (3) the state estimation layer, which enables the framework to predict future object appearance and location. Three different models were trained and tested to analyze the effect of depth along with RGB information. Also, a model is proposed to utilize only depth as a standalone input for tracking purposes. The proposed models were also evaluated in real-time using KinectV2 and showed very promising results. The results of our proposed network structures and their comparison with the state-of-the-art RGB tracking model demonstrate that adding depth significantly improves the accuracy of tracking in a more challenging environment (i.e., cluttered and camouflaged environments). Furthermore, the results of depth-based models showed that depth data can provide enough information for accurate tracking, even without RGB information.

Real-time human detection and tracking in complex environments using single RGBD camera

Detecting and tracking people in real time with RGB-D camera

Multi-modal 3D Human Tracking for Robots in Complex Environment with Siamese Point-Video Transformer

Rethinking Human Pose Estimation for Autonomous Driving with 3D Event Representations.

2D Motion Detection Bounded Hand 3D Trajectory Tracking and Gesture Recognition under Complex Background

Reliably Detecting Humans in Crowded and Dynamic Environments Using RGB-D Camera

Robust 3-D Human Detection in Complex Environments With a Depth Camera.

Human Motion Tracking by Multiple RGBD Cameras.

Robust Real-Time Human Perception with Depth Camera.

Real-time Monocular 3D People Localization and Tracking on Embedded System

RGB-D Human Identification and Tracking in a Smart Environment

Real-time multiple human perception with color-depth cameras on a mobile robot

Deep Attention Models for Human Tracking Using RGBD

Fast Human Detection in RGB-D Images Based on Color-Depth Joint Feature Learning.

Robust Human Detection with Super-Pixel Segmentation and Random Ferns Classification Using RGB-D Camera

A Top-View Multiple People Tracking System Based on Newest YOLOv5 and DeepSort Using Depth Data

An ultra-fast human detection method for color-depth camera

Humans Tracking In The Complicated Background By Multi-Cue Integration

Real-time 3D Deep Multi-Camera Tracking

Reliably Detecting Humans With Rgb-D Camera With Physical Blob Detector Followed By Learning-Based Filtering

Dynamic Visual Localization and Tracking Method Based on Rgb-D Information