Accelerated Evaluation of Automated Vehicles.

Abstract

Automated Vehicles (AVs) must be evaluated thoroughly before their release and deployment. The challenges of AV evaluation stem from two facts. i) Crashes are exceedingly rare events, which makes the Naturalistic-Field Operational Tests (N-FOT) very time-consuming and expensive to conduct. ii) AVs can “cheat” to pass predefined tests. Traditionally, vehicle test protocols are pre-defined and fixed. This is not a problem when the vehicle is “dumb”, but becomes a problem when the vehicle is intelligent and can be customized to excel in the predefined tests. An evaluation approach that represents the real world but not as time-consuming as the N-FOT is needed to address the problems mentioned above.

In this research, we propose an “Accelerated Evaluation” concept to accelerate the evaluations of AV by several orders of magnitude. The interactions between the AV and the surrounding Human-controlled Vehicles (HVs) are modeled based on naturalistic driving database. Four methodologies were developed in this dissertation to form the basis of the Accelerated Evaluation concept. The first method is based on the likelihood analysis of naturalistic driving. The second method provides a solid mathematical basis of the acceleration procedure based on the Importance Sampling theory, such that the statistical equivalence between the accelerated tests and naturalistic driving tests can be guaranteed. The third method, the “Adaptive Accelerated Evaluation”, provides a procedure to find parameters that maximally reduce the test numbers. Finally, the Accelerated Evaluation approach to analyzing the dynamic interactions between AVs and HVs was developed based on stochastic optimization techniques. The proposed approach can be used in simulations, human-in-the-loop tests with driving simulators, hardware-in-the-loop tests, or on-track tests.

Simulation results show that the accelerated tests can reduce the evaluation time of crash, injury or conflict events by 300 to 100,000 times. In other words, driving for 1,000 miles can expose the AV with challenging scenarios that take 300 thousand to 100 million miles in the real-world to encounter. This technique thus has the potential to dramatically reduce the development and validation time of AVs.