Enhanced Parking Perception by Multi-Task Fisheye Cross-view Transformers

Antonyo Musabini,Ivan Novikov,Sana Soula,Christel Leonet,Lihao Wang,Rachid Benmokhtar,Fabian Burger,Thomas Boulay,Xavier Perrotton

2024-09-30

Abstract:Current parking area perception algorithms primarily focus on detecting vacant slots within a limited range, relying on error-prone homographic projection for both labeling and inference. However, recent advancements in Advanced Driver Assistance System (ADAS) require interaction with end-users through comprehensive and intelligent Human-Machine Interfaces (HMIs). These interfaces should present a complete perception of the parking area going from distinguishing vacant slots' entry lines to the orientation of other parked vehicles. This paper introduces Multi-Task Fisheye Cross View Transformers (MT F-CVT), which leverages features from a four-camera fisheye Surround-view Camera System (SVCS) with multihead attentions to create a detailed Bird-Eye View (BEV) grid feature map. Features are processed by both a segmentation decoder and a Polygon-Yolo based object detection decoder for parking slots and vehicles. Trained on data labeled using LiDAR, MT F-CVT positions objects within a 25m x 25m real open-road scenes with an average error of only 20 cm. Our larger model achieves an F-1 score of 0.89. Moreover the smaller model operates at 16 fps on an Nvidia Jetson Orin embedded board, with similar detection results to the larger one. MT F-CVT demonstrates robust generalization capability across different vehicles and camera rig configurations. A demo video from an unseen vehicle and camera rig is available at: <a class="link-external link-https" href="https://streamable.com/jjw54x" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition,Artificial Intelligence

What problem does this paper attempt to address?

The paper attempts to address the issue in parking area perception where existing algorithms mainly rely on homographic projection, which has significant errors, leading to limited detection range and difficulty in accurately identifying the positions and directions of vacant parking spaces and surrounding vehicles. Additionally, current advanced driver assistance systems (ADAS) require more comprehensive and intelligent human-machine interaction (HMI) with users. These interfaces should provide complete parking area perception, including distinguishing the entrance lines of vacant parking spaces and the directions of surrounding vehicles.

Specifically, the paper proposes a Multi-Task Fisheye Cross-View Transformer (MT F-CVT), which utilizes the features of a four-camera fisheye surround-view camera system (SVCS) and a multi-head attention mechanism to generate detailed bird's-eye view (BEV) feature maps. The model detects parking spaces and vehicles through a segmentation decoder and a Polygon-YOLO-based object detection decoder, achieving an average positioning error of 20 centimeters in a 25-meter × 25-meter real open road scenario. Furthermore, the model runs at 16 frames per second on the Nvidia Jetson Orin embedded board, demonstrating strong generalization ability and adaptability to different vehicles and camera configurations.