Abstract:Object detection and semantic segmentation are two fundamental techniques for Intelligent Vehicles (IV) and Advanced Driving Assistance System (ADAS). Motivated by recent studies demonstrating that object detection and semantic segmentation are two highly-correlated tasks, this paper handles the problem of joint object detection and semantic segmentation in traffic scenes. Existing methods perform the joint object detection and semantic segmentation by sharing the same backbone network, but always ignore the interactive connection between the subdividing detection branch and segmentation branch, leading to the insufficient interaction between the two branches. Considering this situation, this paper proposes a joint object detection and semantic segmentation model with the cross-attention and inner-attention mechanisms. The cross-attention mechanism enables to build up the essential interaction between the subdividing detection branch and segmentation branch to fully make use of their correlation. In addition, the inner-attention contributes to strengthening the representations of feature maps in the model. Given an image, an encoder-decoder network is firstly used to extract initial feature maps. Then, the inner-attention mechanism is applied to strengthen the initial feature maps to obtain segmentation feature maps. Subsequently, the cross-attention mechanism utilizes the segmentation feature maps to guide the generation of object detection feature maps. Finally, the semantic segmentation is performed on the segmentation feature maps and object detection is performed on the detection feature maps. In the experiments, two well-known public traffic datasets are used to evaluate our model. Our model achieves the highest performance in comparison with several recently-proposed methods. In addition, some ablation studies are conducted to evaluate the proposed inner-attention and cross-attention mechanisms, and experiment results validate their effectiveness.

Simultaneous End-to-End Vehicle and License Plate Detection With Multi-Branch Attention Neural Network

Scale-Invariant Multidirectional License Plate Detection with the Network Combining Indirect and Direct Branches

Toward End-to-End Car License Plate Detection and Recognition with Deep Neural Networks.

Towards end-to-end car license plate location and recognition in unconstrained scenarios

An End to End Recognition for License Plates Using Convolutional Neural Networks

License Plate Localization in Unconstrained Scenes Using a Two-Stage CNN-RNN

License Plate Detection Using Deep Cascaded Convolutional Neural Networks In Complex Scenes

A Single-Target License Plate Detection with Attention

Vehicle Behavior Recognition using Multi-Stream 3D Convolutional Neural Network

Reading Car License Plates Using Deep Convolutional Neural Networks and LSTMs

License Plate Detection Based on Fully Convolutional Networks

Multi-Oriented and Scale-Invariant License Plate Detection Based on Convolutional Neural Networks

Lightweight Fully Convolutional Network for License Plate Detection

A Joint Object Detection and Semantic Segmentation Model with Cross-Attention and Inner-Attention Mechanisms

Multiclass objects detection algorithm using DarkNet-53 and DenseNet for intelligent vehicles

License Plate Recognition Using Deep FCN.

Effective deep neural networks for license plate detection and recognition

Recognition of vehicle license plates in highway scenes with deep fusion network and connectionist temporal classification

Reading Car License Plates Using Deep Neural Networks.

A CNN-Based Approach for Automatic License Plate Recognition in the Wild

Attention-Mechanism-based Tracking Method for Intelligent Internet of Vehicles