Abstract:The ability to perceive humans is an essential requirement for safe and efficient human-robot interaction. In real-world applications, the need for a robot to interact in real time with multiple humans in a dynamic, 3-D environment presents a significant challenge. The recent availability of commercial color-depth cameras allow for the creation of a system that makes use of the depth dimension, thus enabling a robot to observe its environment and perceive in the 3-D space. Here we present a system for 3-D multiple human perception in real time from a moving robot equipped with a color-depth camera and a consumer-grade computer. Our approach reduces computation time to achieve real-time performance through a unique combination of new ideas and established techniques. We remove the ground and ceiling planes from the 3-D point cloud input to separate candidate point clusters. We introduce the novel information concept, depth of interest, which we use to identify candidates for detection, and that avoids the computationally expensive scanning-window methods of other approaches. We utilize a cascade of detectors to distinguish humans from objects, in which we make intelligent reuse of intermediary features in successive detectors to improve computation. Because of the high computational cost of some methods, we represent our candidate tracking algorithm with a decision directed acyclic graph, which allows us to use the most computationally intense techniques only where necessary. We detail the successful implementation of our novel approach on a mobile robot and examine its performance in scenarios with real-world challenges, including occlusion, robot motion, nonupright humans, humans leaving and reentering the field of view (i.e., the reidentification challenge), human-object and human-human interaction. We conclude with the observation that the incorporation of the depth information, together with the use of modern techniques in new ways, we are able to create an accurate system for real-time 3-D perception of humans by a mobile robot.

Real-time human detection and tracking in complex environments using single RGBD camera

Detecting and tracking people in real time with RGB-D camera

Multi-modal 3D Human Tracking for Robots in Complex Environment with Siamese Point-Video Transformer

Rethinking Human Pose Estimation for Autonomous Driving with 3D Event Representations.

2D Motion Detection Bounded Hand 3D Trajectory Tracking and Gesture Recognition under Complex Background

Reliably Detecting Humans in Crowded and Dynamic Environments Using RGB-D Camera

Robust 3-D Human Detection in Complex Environments With a Depth Camera.

Human Motion Tracking by Multiple RGBD Cameras.

Robust Real-Time Human Perception with Depth Camera.

Real-time Monocular 3D People Localization and Tracking on Embedded System

RGB-D Human Identification and Tracking in a Smart Environment

Real-time multiple human perception with color-depth cameras on a mobile robot

Tetracyclines in treatment of rheumatoid arthritis

Fast Human Detection in RGB-D Images Based on Color-Depth Joint Feature Learning.

Robust Human Detection with Super-Pixel Segmentation and Random Ferns Classification Using RGB-D Camera

A Top-View Multiple People Tracking System Based on Newest YOLOv5 and DeepSort Using Depth Data

An ultra-fast human detection method for color-depth camera

Humans Tracking In The Complicated Background By Multi-Cue Integration

Real-time 3D Deep Multi-Camera Tracking

Reliably Detecting Humans With Rgb-D Camera With Physical Blob Detector Followed By Learning-Based Filtering

Dynamic Visual Localization and Tracking Method Based on Rgb-D Information