Abstract:Purpose This paper aims to develop a robust person tracking method for human following robots. The tracking system adopts the multimodal fusion results of millimeter wave (MMW) radars and monocular cameras for perception. A prototype of human following robot is developed and evaluated by using the proposed tracking system. Design/methodology/approach Limited by angular resolution, point clouds from MMW radars are too sparse to form features for human detection. Monocular cameras can provide semantic information for objects in view, but cannot provide spatial locations. Considering the complementarity of the two sensors, a sensor fusion algorithm based on multimodal data combination is proposed to identify and localize the target person under challenging conditions. In addition, a closed-loop controller is designed for the robot to follow the target person with expected distance. Findings A series of experiments under different circumstances are carried out to validate the fusion-based tracking method. Experimental results show that the average tracking errors are around 0.1 m. It is also found that the robot can handle different situations and overcome short-term interference, continually track and follow the target person. Originality/value This paper proposed a robust tracking system with the fusion of MMW radars and cameras. Interference such as occlusion and overlapping are well handled with the help of the velocity information from the radars. Compared to other state-of-the-art plans, the sensor fusion method is cost-effective and requires no additional tags with people. Its stable performance shows good application prospects in human following robots.

Human Tracking System Based on Adaptive Multi-Feature Mean-Shift for Robot under the Double-Layer Locating Mechanism

Human Tracking and Following in Dynamic Environment for Service Robots

Beyond Traditional Driving Scenes: A Robotic-Centric Paradigm for 2D+3D Human Tracking Using Siamese Transformer Network

People Following System Based on LRF

Multi-modal 3D Human Tracking for Robots in Complex Environment with Siamese Point-Video Transformer

Accurate and Real-Time 3-D Tracking for the Following Robots by Fusing Vision and Ultrasonar Information

An Efficient and Robust System for Human Following Scenario Using Differential Robot

Human tracking of robots based on double-layer cooperative locating mechanism

Human Tracking Based on Multi-Feature for Intelligent Robot Under the CTF Locating Strategy

A Skeleton and Visual Tracking Fusion Based Person-Following System for Mobile Service Robots

Person Tracking of Mobile Robot Using Improved Mean Shift

A novel tracking system for human following robots with fusion of MMW radar and monocular vision

Real-time Target Tracking System for Person-Following Robot

Advanced Multi-Sensor Person-Following System on a Mobile Robot: Design, Construction and Measurements

Human Tracking Method Based On Multi-Template Color-Texture Mean-Shift Algorithm

Object Tracking by Multi-Degrees of Freedom Mean Shift Procedure Combined with the Kalman Particle Filter Algorithm

Human tracking based on adaptive multi-feature mean-shift algorithm

Person Tracking System by Fusing Multicues Based on Patches

Person following of binocular robot by extracting multiple features

A Target Tracking Algorithm Based On Mean Shift With Feature Fusion

Sensor Integration for Person Tracking and Following with Mobile Robot