Accurate, Low-Latency Visual Perception for Autonomous Racing:Challenges, Mechanisms, and Practical Solutions

Kieran Strobel,Sibo Zhu,Raphael Chang,Skanda Koppula

DOI: https://doi.org/10.48550/arXiv.2007.13971

2020-07-28

Abstract:Autonomous racing provides the opportunity to test safety-critical perception pipelines at their limit. This paper describes the practical challenges and solutions to applying state-of-the-art computer vision algorithms to build a low-latency, high-accuracy perception system for DUT18 Driverless (DUT18D), a 4WD electric race car with podium finishes at all Formula Driverless competitions for which it raced. The key components of DUT18D include YOLOv3-based object detection, pose estimation, and time synchronization on its dual stereovision/monovision camera setup. We highlight modifications required to adapt perception CNNs to racing domains, improvements to loss functions used for pose estimation, and methodologies for sub-microsecond camera synchronization among other improvements. We perform a thorough experimental evaluation of the system, demonstrating its accuracy and low-latency in real-world racing scenarios.

Computer Vision and Pattern Recognition,Robotics

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to implement a high - precision and low - latency visual perception system in autonomous racing cars. Specifically, the paper focuses on how to test the safety - critical perception pipeline of self - driving cars under extreme conditions, and proposes the challenges, mechanisms and practical solutions for the low - latency and high - precision visual perception system of the DUT18 Driverless racing car. The main problems include:

1. **Reducing the latency of the visual perception system**: The visual perception system dominates the overall latency in self - driving cars. Especially in high - speed racing cars, latency directly affects the vehicle's safety and response speed. Therefore, how to design a low - latency visual perception system is a core issue.

2. **Improving positioning accuracy**: How to ensure the high - precision of the visual perception system under different weather and lighting conditions, especially in outdoor environments, is another important research point.

3. **Adapting to the racing environment**: How to adapt existing computer vision algorithms (such as YOLOv3) to the racing environment, including technical details such as improvement of the loss function and time synchronization of multiple cameras, is also the focus of the paper's exploration.

4. **Balancing hardware and software**: While selecting an appropriate hardware configuration (such as lens focal length, sensor pixel size), how to improve performance through software optimization to meet the requirements of system efficiency, accuracy and coverage.

By describing these problems and their solutions in detail, the paper aims to provide a comprehensive design guide for the visual perception system of high - performance self - driving cars.