Abstract:Pedestrian detection has always been a research hotspot in the Advanced Driving Assistance System (ADAS) with great progress in recent years. However, for the ADAS, we not only need to detect the behavior of pedestrians in front of the vehicle but also predict future action and the motion trajectory. Therefore, in this paper, we propose a human key point combined optical flow network (KPOF-Net) in the vehicle ADAS for the occlusion situation in the actual scene. When the vehicle encounters a blocked pedestrian at a traffic intersection, we used self-flow to estimate the global optical flow in the image sequence and then proposed a White Edge Cutting (WEC) algorithm to remove obstructions and simply modified the generative adversarial network to initialize pedestrians behind the obstructions. Next, we extracted pedestrian optical flow information and human joint point information in parallel, among which we trained four human key point models suitable for traffic intersections. At last, KPOF-GPDM fusion was proposed to predict the future status and walking trajectories of pedestrians, which combined optical flow information with human key point information. In the experiment, we did not merely compare our method with other four representative approaches in the same scene sequences. We also verified the accuracy of the pedestrian motion state and motion trajectory prediction of the system after fusion of human joint points and optical flow information. Taking into account the real-time performance of the system, in the low-speed and barrier-free environment, the comparative analysis only uses optical flow information, human joint point information, and KPOF-Net three prediction models. The results show that (1) in the same traffic environment, our proposed KPOF-Net can predict the change of pedestrian motion state about 5 frames (about 0.26 s) ahead of other excellent systems; (2) at the same time, our system predicts the trajectory of the pedestrian more accurately than the other four systems, which can achieve more stable minimum error ±0.04 m; (3) in a low-speed, barrier-free experimental environment, our proposed trajectory prediction model that integrates human joint points and optical flow information has higher prediction accuracy and smaller fluctuations than a single-information prediction model, and it can be well applied to automobiles’ ADAS.

Optical Flow-Based Gait Modeling Algorithm for Pedestrian Navigation Using Smartphone Sensors

Optical Flow Based Step Length Estimation for Indoor Pedestrian Navigation on a Smartphone

Pedestrian Gait Information Aided Visual Inertial SLAM for Indoor Positioning Using Handheld Smartphones

Pedestrian movement trajectory reappearance and crossing feature expression based on video processing

Pedestrian Navigation Method Based on Machine Learning and Gait Feature Assistance

Accurately Counting Steps Of The Pedestrian With Varying Walking Speeds

A Pose Awareness Solution For Estimating Pedestrian Walking Speed

Plastic Optical Fiber Integrated with Smartphone for Gait Monitoring

Dense optical flow-based gait recognition

At the Traffic Intersection, Stopping, or Walking? Pedestrian Path Prediction Based on KPOF-GPDM for Driving Assistance

A Learning Method for Pedestrian Activity Classification

Adaptive Kalman Filtering-Based Pedestrian Navigation Algorithm for Smartphones

Improvement of optical flow in pedestrian detection based on pictorial structure

On the Pedestrian Flow Analysis Through Passive WiFi Sensing

Incremental Learning for Video-Based Gait Recognition with LBP Flow

RPNOS: Reliable Pedestrian Navigation on a Smartphone

Continuous Motion Recognition for Natural Pedestrian Dead Reckoning Using Smartphone Sensors

An Indoor Pedestrian Positioning Algorithm Based on Smartphone Sensor

An Effective Algorithm to Simulate Pedestrian Flow Using the Heuristic Force-Based Model

A Novel Walking Detection and Step Counting Algorithm Using Unconstrained Smartphones

A novel heading estimation algorithm for pedestrian using a smartphone without attitude constraints