A Theoretical Foundation of Intelligence Testing and Its Application for Intelligent Vehicles

Li Li,Nanning Zheng,Fei-Yue Wang

DOI: https://doi.org/10.1109/tits.2020.2991039

IF: 8.5

2021-10-01

IEEE Transactions on Intelligent Transportation Systems

Abstract:Intelligent vehicle testing received quickly increasing attention due to the intermittent accidents of intelligent vehicle prototypes that occurred recently. In this paper, we investigate the theoretical underpinnings of such testing and establish a rigid analyzing framework for general intelligence testing problems by borrowing the ideas of Probably Approximately Correct (PAC) learning. Our focus is on the relationship between the number of sampled scenarios and the testing efficiency. We explain various existing algorithms within this new framework and clarify some misconceptions about the reasoning underpinning these methods. We show that intelligent vehicles are testable if the testing scenarios are well defined and appropriately sampled. Moreover, we propose a sampling strategy to generate new challenging scenarios to boost testing efficiency.

engineering, electrical & electronic,transportation science & technology, civil

What problem does this paper attempt to address?

The paper aims to address the theoretical foundation issues in intelligent vehicle testing and proposes a method based on the Probably Approximately Correct (PAC) learning framework to evaluate the capabilities of intelligent vehicles. Specifically, the paper attempts to answer the following core questions:

1. **Is there a solid theoretical foundation for intelligent vehicle testing?** The paper clarifies some ambiguous concepts (such as "scenarios" or "sampling") by introducing the PAC testing framework and provides theoretical support.

2. **What are the key differences between performance testing of intelligent vehicles and current intelligent testing?** The paper distinguishes between traditional vehicle performance testing and intelligent testing of intelligent vehicles, emphasizing that intelligent testing needs to handle more complex scenarios.

3. **Do existing intelligent vehicle testing methods overlook some difficulties?** The paper points out some issues that existing methods may overlook and proposes improvements.

4. **Is it possible to efficiently and reliably test the capabilities of intelligent vehicles using a limited number of sampled scenarios?** The paper demonstrates that if the test scenarios are properly defined and sampled appropriately, intelligent vehicles can be effectively tested.

In addition, the paper explores how to generate new challenging scenarios to improve testing efficiency and discusses the impact of active sampling on test effectiveness. By introducing the PAC testing framework, the paper provides a rigorous analytical framework for intelligent vehicle testing and explains the working principles of various existing algorithms, clarifying the misunderstandings behind these methods.