Abstract:Human pose estimation is a key technique for many vision-based mobile applications. Yet existing multi-person pose-estimation methods fail to achieve a satisfactory user experience on commodity mobile devices such as smartphones, due to their long model-inference latency. In this paper, we propose MobiPose, a system designed to enable real-time multi-person pose estimation on mobile devices through three novel techniques. First, MobiPose takes a motion-vector-based approach to fast locate the human proposals across consecutive frames by fine-grained tracking of joints of human body, rather than running the expensive human-detection model for every frame. Second, MobiPose designs a mobile-friendly model that uses lightweight multi-stage feature extractions to significantly reduce the latency of pose estimation without compromising the model accuracy. Third, MobiPose leverages the heterogeneous computing resources of both CPU and GPU to execute the pose estimation model for multiple persons in parallel, which further reduces the total latency. We have implemented the MobiPose system on off-the-shelf commercial smartphones and conducted comprehensive experiments to evaluate the effectiveness of the proposed techniques. Evaluation results show that MobiPose achieves over 20 frames per second pose estimation with 3 persons per frame, and significantly outperforms the state-of-the-art baseline, with a speedup of up to 4.5X and 2.8X in latency on CPU and GPU, respectively, and an improvement of 5.1% in pose-estimation model accuracy. Furthermore, MobiPose achieves up to 62.5% and 37.9% energy-per-frame saving on average in comparison with the baseline on mobile CPU and GPU, respectively.

Real-time Pose Estimation for Mobile Devices: A Review

DIR-BHRNet: A Lightweight Network for Real-time Vision-based Multi-person Pose Estimation on Smartphones

A Review on Human Pose Estimation

Efficient Monocular Human Pose Estimation Based on Deep Learning Methods: A Survey

Bottom-up approaches for multi-person pose estimation and it's applications: A brief review

MovePose: A High-performance Human Pose Estimation Algorithm on Mobile and Edge Devices

MVPose: Realtime Multi-Person Pose Estimation Using Motion Vector on Mobile Devices

OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields

Deep Learning-Based Human Pose Estimation: A Survey

A Systematic Review of the Application of Camera-Based Human Pose Estimation in the Field of Sport and Physical Exercise

Vision-Based Human Pose Estimation via Deep Learning: A Survey

EffiPoseNet: Integrating Ghost Modules and CoordAttention for Efficient Real-Time Human Pose Estimation on Mobile Devices

MobiPose: real-time multi-person pose estimation on mobile devices

Deep learning-based human body pose estimation in providing feedback for physical movement: A review

Comparative Analysis of Skeleton-Based Human Pose Estimation

A Real-time Human Pose Estimation Approach for Optimal Sensor Placement in Sensor-based Human Activity Recognition

Human Pose Detection for Robotic-Assisted and Rehabilitation Environments

An RGB-D Based Approach for Human Pose Estimation

Markerless human pose estimation for biomedical applications: a survey

An enhanced real-time human pose estimation method based on modified YOLOv8 framework