Abstract:• Pioneering research on traffic command recognition distinguishing directions and gestures. • A two-stage recognition model exploiting skeletal geometry and co-occurrence features. • A specialized dataset for recognizing Chinese traffic commands at road intersections. Understanding traffic officer commands is a fundamental perception task for intelligent vehicles in driver assistance and autonomous driving. Previous studies have emphasized explicit traffic command gesture recognition but have not considered situations where the traffic officer is controlling the subjects in other directions, which would also influence decision-making of the ego vehicle. To fill in the gap, this article aims to research visual skeleton-based recognition of traffic commands occurring at road intersections, where both command directions and gestures should be determined. Specifically, a two-stage recognition framework for four cross-shaped directions and eight command gestures is proposed. Two kinds of handcrafted features, including upper-body geometric features and keypoint co-occurrence features, are established with estimated 2D human keypoint coordinates and heatmaps and further combined into a deep learning network. The first stage handles human body orientation classification, while the second stage addresses command gesture recognition with extra usage of the output from the first stage. Combining the recognized body orientation and command gesture, the type of traffic command can ultimately be inferred. For training and validation, a dataset termed the Chinese Traffic Command at Intersections (CTCX) is built. The proposed method gains an outperforming edit accuracy of 89.67% on the CTCX test set, demonstrating its effectiveness. This work provides a foundation in this area and is expected to inspire more research on traffic command recognition with directions in the near future.

Dual-module spatial temporal information enhancement graph convolutional network for recognizing traffic police command gestures

Chinese Traffic Police Gesture Recognition Based on Graph Convolutional Network in Natural Scene

Traffic Police 3D Gesture Recognition Based on Spatial–Temporal Fully Adaptive Graph Convolutional Network

Traffic police command gesture recognition technology based on machine vision and two-stream spatio-temporal attention graph convolutional network

Attention Mechanism Based on Improved Spatial-Temporal Convolutional Neural Networks for Traffic Police Gesture Recognition

Visual Recognition of traffic police gestures with convolutional pose machine and handcrafted features

Simple But Effective: Upper-Body Geometric Features for Traffic Command Gesture Recognition

Low light recognition of traffic police gestures based on lightweight extraction of skeleton features

FFCSLT: a deep learning model for traffic police hand gesture recognition using surface electromyographic signals

Traffic Police Gesture Command Recognition Based on Convolutional Neural Network AlexNet

An Attentional Spatial Temporal Graph Convolutional Network with Co-Occurrence Feature Learning for Action Recognition

Gesture recognition of traffic police based on static and dynamic descriptor fusion

mm-TPG: Traffic Policemen Gesture Recognition Based on Millimeter Wave Radar Point Cloud

Max-covering scheme for gesture recognition of Chinese traffic police

Multimodal Spatiotemporal Feature Map for Dynamic Gesture Recognition

Skeleton-based Traffic Command Recognition at Road Intersections for Intelligent Vehicles

A Spatio-Temporal Graph Convolutional Network for Gesture Recognition from High-Density Electromyography

STCN-GR: Spatial-Temporal Convolutional Networks for Surface-Electromyography-Based Gesture Recognition

Spatio-Temporal Dynamic Attention Graph Convolutional Network Based on Skeleton Gesture Recognition

A Spatio-Temporal Multilayer Perceptron for Gesture Recognition

Searching Multi-Rate and Multi-Modal Temporal Enhanced Networks for Gesture Recognition