Unsafe At Any Level: NHTSA's levels of automation are a liability for autonomous vehicle design and regulation

Marc Canellas,Rachel Haga
DOI: https://doi.org/10.1145/3342102
2020-03-01
Abstract:Walter Huang, a 38-year-old Apple Inc. engineer, died on March 23, 2018, after his Tesla Model X crashed into a highway barrier in Mountain View, California. Tesla immediately disavowed responsibility for the accident. "The fundamental premise of both moral and legal liability is a broken promise, and there was none here: [Mr. Huang] was well aware that the Autopilot was not perfect [and the] only way for this accident to have occurred is if Mr. Huang was not paying attention to the road, despite the car providing multiple warnings to do so." This is the standard response from Tesla and Uber, the manufacturers of the automated vehicles involved in the six fatal accidents to date: the automated vehicle isn't perfect, the driver knew it wasn't perfect, and if only the driver had been paying attention and heeded the vehicle's warnings, the accident would never have occurred. However, as researchers focused on human-automation interaction in aviation and military operations, we cannot help but wonder if there really are no broken promises and no legal liabilities. Science has a critical role in determining legal liability, and courts appropriately rely on scientists and engineers to determine whether an accident, or harm, was foreseeable. Specifically, a designer could be found liable if, at the time of the accident, scientists knew there was a systematic relationship between the accident and the designer's untaken precaution. Nearly 70 years of research provides an undeniable answer: It is insufficient, inappropriate, and dangerous to automate everything you can and leave the rest to the human. There is a systematic relationship between the design of automated vehicles and the types of accidents that are occurring now and will inevitably continue to occur in the future. These accidents were not unforeseeable and the drivers were not exclusively to blame.
Computers and Society
What problem does this paper attempt to address?
The problem that this paper attempts to solve is the legal liability issue in the design and regulation of autonomous vehicles. Specifically, the author explores the applicability and potential flaws of the Levels of Automation (LOA) framework defined by the National Highway Traffic Safety Administration (NHTSA) in the design of autonomous vehicles and in the formulation of regulations. The paper points out that the current LOA framework focuses too much on the capabilities of the automated system and ignores the complexity of human - machine collaboration and the needs of human operators, which may lead to unforeseen accidents and the ambiguity of legal liability. The author believes that, based on existing scientific research on human - machine interaction, it can be foreseen that design flaws in autonomous vehicles may lead to accidents, so the LOA framework needs to be reconsidered and improved to better ensure road safety and clarify legal liability. By analyzing several fatal accident cases involving autonomous vehicles, especially those of Tesla vehicles, the paper points out several major problems under the current LOA framework: 1. **Irrational task allocation between the automated system and human operators**: The LOA framework assumes that the task allocation between the automated system and human operators is clear and rational, but in fact, this allocation often ignores the capabilities and needs of human operators, resulting in human operators being unable to take over control in a timely manner in emergency situations. 2. **The monitoring task of the automated system is too difficult for human operators**: Research shows that long - term monitoring of the automated system will cause human operators to lose real - time situational awareness, and thus be unable to react quickly in emergency situations. 3. **The LOA framework lacks support for human - machine collaboration**: The LOA framework focuses more on "who does which tasks" rather than on how to support human operators to work effectively in the automated system. This design flaw may cause human operators to be unable to effectively intervene or take over the system at critical moments. 4. **Unclear legal liability**: When an accident occurs, manufacturers usually shift the blame to the driver, claiming that the driver has not monitored the system as required. However, from the perspective of human - machine interaction science, this division of liability is unfair and unreasonable. In summary, the main purpose of the paper is to reveal the deficiencies of the current LOA framework in the design and regulation of autonomous vehicles and call on relevant institutions and manufacturers to re - examine and improve this framework to ensure the safety and reliability of autonomous driving technology.