Abstract:• Pioneering research on traffic command recognition distinguishing directions and gestures. • A two-stage recognition model exploiting skeletal geometry and co-occurrence features. • A specialized dataset for recognizing Chinese traffic commands at road intersections. Understanding traffic officer commands is a fundamental perception task for intelligent vehicles in driver assistance and autonomous driving. Previous studies have emphasized explicit traffic command gesture recognition but have not considered situations where the traffic officer is controlling the subjects in other directions, which would also influence decision-making of the ego vehicle. To fill in the gap, this article aims to research visual skeleton-based recognition of traffic commands occurring at road intersections, where both command directions and gestures should be determined. Specifically, a two-stage recognition framework for four cross-shaped directions and eight command gestures is proposed. Two kinds of handcrafted features, including upper-body geometric features and keypoint co-occurrence features, are established with estimated 2D human keypoint coordinates and heatmaps and further combined into a deep learning network. The first stage handles human body orientation classification, while the second stage addresses command gesture recognition with extra usage of the output from the first stage. Combining the recognized body orientation and command gesture, the type of traffic command can ultimately be inferred. For training and validation, a dataset termed the Chinese Traffic Command at Intersections (CTCX) is built. The proposed method gains an outperforming edit accuracy of 89.67% on the CTCX test set, demonstrating its effectiveness. This work provides a foundation in this area and is expected to inspire more research on traffic command recognition with directions in the near future.

Max-covering scheme for gesture recognition of Chinese traffic police

Dynamic hand gesture recognition using hidden Markov models

Gesture recognition of traffic police based on static and dynamic descriptor fusion

Visual Recognition of traffic police gestures with convolutional pose machine and handcrafted features

Chinese Traffic Police Gesture Recognition Based on Graph Convolutional Network in Natural Scene

Real-time Dynamic Gesture Recognition and Hand Servo Tracking Using PTZ Camera

2D Motion Detection Bounded Hand 3D Trajectory Tracking and Gesture Recognition under Complex Background

A real-time hand gesture recognition method

Intelligent Video Surveillance for Checking Attendance of Traffic Controllers in Level Crossing

Dual-module spatial temporal information enhancement graph convolutional network for recognizing traffic police command gestures

Simple But Effective: Upper-Body Geometric Features for Traffic Command Gesture Recognition

Attention Mechanism Based on Improved Spatial-Temporal Convolutional Neural Networks for Traffic Police Gesture Recognition

Traffic Police 3D Gesture Recognition Based on Spatial–Temporal Fully Adaptive Graph Convolutional Network

mm-TPG: Traffic Policemen Gesture Recognition Based on Millimeter Wave Radar Point Cloud

Traffic police command gesture recognition technology based on machine vision and two-stream spatio-temporal attention graph convolutional network

Low light recognition of traffic police gestures based on lightweight extraction of skeleton features

Parsing the Hand Gesture of Traffic Police Officers by Using OpenPose

Skeleton-based Traffic Command Recognition at Road Intersections for Intelligent Vehicles

Gesture Detection and Recognition Based on Object Detection in Complex Background

Gesture Recognition Using Enhanced Depth Motion Map and Static Pose Map

Detection of hand-raising gestures based on body silhouette analysis