Abstract:3D human tracking plays a crucial role in the automation intelligence system. Current approaches focus on achieving higher performance on traditional driving datasets like KITTI, which overlook the jitteriness of the platform and the complexity of the environments. Once the scenarios are migrated to jolting robot platforms, they all degenerate severely with only a 20-60% success rate, which greatly restricts the high-level application of autonomous systems. In this work, beyond traditional flat scenes, we introduce Multi-modal Human Tracking Paradigm (MHTrack), a unified multimodal transformer-based model that can effectively track the target person frame-by-frame in point and video sequences. Specifically, we design a speed-inertia module-assisted stabilization mechanism along with an alternate training strategy to better migrate the tracking algorithm to the robot platform. To capture more target-aware information, we combine the geometric and appearance features of point clouds and video frames together based on a hierarchical Siamese Transformer Network. Additionally, considering the prior characteristics of the human category, we design a lateral cross-attention pyramid head for deeper feature aggregation and final 3D BBox generation. Extensive experiments confirm that MHTrack significantly outperforms the previous state-of-the-arts on both open-source datasets and large-scale robotic datasets. Further analysis verifies each component’s effectiveness and shows the robotic-centric paradigm’s promising potential when deployed into dynamic robotic systems.

A Recursive Framework for Automatic Face Tracking

Robust Human Tracking Via Key Face Information.

A Simple 3d Face Tracking Method Based On Depth Information

Beyond Traditional Driving Scenes: A Robotic-Centric Paradigm for 2D+3D Human Tracking Using Siamese Transformer Network

Accurate and Real-Time 3-D Tracking for the Following Robots by Fusing Vision and Ultrasonar Information

REAL TIME FACE TRACKING IN EMBEDDED ENVIRONMENT

Robust Object Tracking with a Hierarchical Ensemble Framework

Robust Real-Time Face Tracking And Modeling From Video

An Effective Approach to 3D Deformable Surface Tracking

Real-Time Facial Features Tracker With Motion Estimation And Feedback

Research on face tracking Algorithm Based on Detection and Supervision Tracking

Long-term face tracking in the wild using deep learning

Displaced Dynamic Expression Regression for Real-Time Facial Tracking and Animation

Robust 3D face modeling and tracking from RGB-D images

3D Face Tracking from 2D Video through Iterative Dense UV to Image Flow

A Multi-body Tracking Framework - From Rigid Objects to Kinematic Structures

Facial Feature Tracking Combining Model-Based and Model-Free Method

Long-Term Face Tracking for Crowded Video-Surveillance Scenarios

Dynamic High Resolution Deformable Articulated Tracking

3D Multi-Object Tracking with Adaptive Cubature Kalman Filter for Autonomous Driving

Real-Time Face Alignment With Tracking In Video