Real-Time Traffic Light Recognition with Lightweight State Recognition and Ratio-Preserving Zero Padding

Jihwan Choi,Harim Lee
DOI: https://doi.org/10.3390/electronics13030615
IF: 2.9
2024-02-01
Electronics
Abstract:As online shopping is becoming mainstream, driven by the social impact of Coronavirus disease-2019 (COVID-19) as well as the development of Internet services, the demand for autonomous delivery mobile robots is rapidly increasing. This trend has brought the autonomous mobile robot market to a new turning point, with expectations that numerous mobile robots will be driving on roads with traffic. To achieve these expectations, autonomous mobile robots should precisely perceive the situation on roads with traffic. In this paper, we revisit and implement a real-time traffic light recognition system with a proposed lightweight state recognition network and ratio-preserving zero padding, which is a two-stage system consisting of a traffic light detection (TLD) module and a traffic light status recognition (TLSR) module. For the TLSR module, this work proposes a lightweight state recognition network with a small number of weight parameters, because the TLD module needs more weight parameters to find the exact location of traffic lights. Then, the proposed effective and lightweight network architecture is constructed by using skip connection, multifeature maps with different sizes, and kernels of appropriately tuned sizes. Therefore, the network has a negligible impact on the overall processing time and minimal weight parameters while maintaining high performance. We also propose to utilize a ratio-preserving zero padding method for data preprocessing for the TLSR module to enhance recognition accuracy. For the TLD module, extensive evaluations with varying input sizes and backbone network types are conducted, and then appropriate values for those factors are determined, which strikes a balance between detection performance and processing time. Finally, we demonstrate that our traffic light recognition system, utilizing the TLD module's determined parameters, the proposed network architecture for the TLSR module, and the ratio-preserving zero padding method can reliably detect the location and state of traffic lights in real-world videos recorded in Gumi and Deagu, Korea, while maintaining at least 30 frames per second for real-time operation.
engineering, electrical & electronic,computer science, information systems,physics, applied
What problem does this paper attempt to address?
The main aim of this paper is to address the following issues: ### Paper Objectives - **Develop a real-time traffic signal recognition system**: Given the rise of online shopping and the resulting increased demand for autonomous delivery robots, the paper aims to develop a system capable of real-time traffic signal recognition. This system is specifically designed for autonomous mobile robots operating on roads to ensure they can accurately perceive the status of traffic signals in their surroundings. - **Propose a lightweight state recognition network**: To improve the performance of the traffic signal light state recognition (TLSR) module and reduce the number of its weight parameters, the paper proposes a lightweight and efficient network architecture. This network is constructed using skip connections, multi-scale feature maps, and appropriately adjusted convolution kernel sizes, thereby maintaining high performance with minimal impact on overall processing time. - **Adopt the ratio-preserving zero-padding method**: To further enhance the performance of the TLSR module, the paper also proposes a ratio-preserving zero-padding (RZP) data preprocessing method. This method pads the traffic signal light images while preserving their aspect ratio to avoid image distortion caused by scaling. - **Optimize the traffic signal light detection module**: To achieve the above goals, the paper also thoroughly investigates the traffic signal light detection (TLD) module, including the evaluation of different input image sizes and backbone network types to find a balance between detection performance and processing time. ### Solution Overview 1. **Two-stage system architecture**: The system consists of two key modules—the TLD module, which is responsible for detecting the positions of traffic signals in images, and the TLSR module, which is used to recognize the states of the detected traffic signals. 2. **TLSR module**: A lightweight network architecture is proposed, utilizing advanced network structure features such as Feature Pyramid Network (FPN) and residual connections to reduce the number of weight parameters and improve recognition performance. Additionally, by using the RZP method, the image ratio can be effectively preserved, thereby enhancing recognition accuracy. 3. **TLD module**: This module is based on deep learning technology, using an Efficient Decoupled Head (EDH) to predict the position information of traffic signals. Furthermore, by comprehensively evaluating the effects of different input image sizes and backbone network types, the paper determines the optimal parameter configuration suitable for real-time operation. Through these methods, the paper proposes a system capable of reliably detecting and recognizing the positions and states of traffic signals in real-world videos, and it can operate at a speed of at least 30 frames per second, meeting the requirements for real-time operation.