Abstract:High-performing, real-time pose detection and tracking in real-time will enable computers to develop a finer-grained and more natural understanding of human behavior. However, the implementation of real-time human pose estimation remains a challenge. On the one hand, the performance of semantic keypoint tracking in live video footage requires high computational resources and large parameters, which limiting the accuracy of pose estimation. On the other hand, some transformer-based models were proposed recently with outstanding performance and much fewer parameters and FLOPs. However, the self-attention module in the transformer is not computationally friendly, which makes it difficult to apply these excellent models to real-time jobs. To overcome the above problems, we propose a transformer-like model, named ShiftPose, which is regression-based approach. The ShiftPose does not contain any self-attention module. Instead, we replace the self-attention module with a non-parameter operation called the shift operator. Meanwhile, we adapt the bridge-branch connection, instead of a fully-branched connection, such as HRNet, as our multi-resolution integration scheme. Specifically, the bottom half of our model adds the previous output, as well as the output from the top half of our model, corresponding to its resolution. Finally, the simple, yet promising, disentangled representation (SimDR) was used in our study to make the training process more stable. The experimental results on the MPII datasets were 86.4 PCKH, 29.1PCKH@0.1. On the COCO dataset, the results were 72.2 mAP and 91.5 AP50, 255 fps on GPU, with 10.2M parameters, and 1.6 GFLOPs. In addition, we tested our model for single-stage 3D human pose estimation and draw several useful and exploratory conclusions. The above results show good performance, and this paper provides a new method for high-performance, real-time attitude detection and tracking.

Multi-Person Pose Estimation on Embedded Device

MCMC: Multi-Constrained Model Compression Via One-Stage Envelope Reinforcement Learning.

Extendable Multi-Device Collaborative Pipeline Parallel Inference in the Edge-Cloud Scenario

Dual Regression for Efficient Hand Pose Estimation

A Compression Pipeline for One-Stage Object Detection Model

Lite Pose: Efficient Architecture Design for 2D Human Pose Estimation

Lightweight Architecture for Real-Time Hand Pose Estimation with Deep Supervision

FCN-Pose: A Pruned and Quantized CNN for Robot Pose Estimation for Constrained Devices

Efficient 2.5D Hand Pose Estimation via Auxiliary Multi-Task Training for Embedded Devices

EfficientPose: Scalable single-person pose estimation

Making DensePose fast and light

Energy-efficient Deployment of Deep Learning Applications on Cortex-M based Microcontrollers using Deep Compression

Adaptive Deep Learning for Efficient Visual Pose Estimation aboard Ultra-low-power Nano-drones

To Compress, or Not to Compress: Characterizing Deep Learning Model Compression for Embedded Inference

FastDeepIoT: Towards Understanding and Optimizing Neural Network Execution Time on Mobile and Embedded Devices

Deep Model Compression and Architecture Optimization for Embedded Systems: A Survey

Pose-native Network Architecture Search for Multi-person Human Pose Estimation

DIR-BHRNet: A Lightweight Network for Real-time Vision-based Multi-person Pose Estimation on Smartphones

Human Pose Estimation from Depth Images via Inference Embedded Multi-task Learning

A Deconvolutional Bottom-up Deep Network for Multi-Person Pose Estimation.

Shift Pose: A Lightweight Transformer-like Neural Network for Human Pose Estimation