Automated Vehicle Crash Rate Comparison Using Naturalistic Data

January 8, 2016

Self-driving cars are quickly moving from prototype to everyday reality. During this transition, the question that is first and foremost on the mind of the public and policy makers is whether or not self-driving cars are more prone to crashes. This would seem to be an easy question to answer: simply compare current published crash rates with the data on self-driving cars. A deeper look at the available data and collection methodologies, however, reveals that such a simple comparison is problematic.

Two factors complicate the national crash data. First, states have different requirements concerning what incidents are reported as crashes. Second, many crashes go unreported. Estimates of unreported rates of crashes have ranged from as little as 15.4 percent to as much as 59.7 percent (Blincoe et al., 2015; M. Davis & Co, 2015). The result is that the current national crash rate is essentially a low estimate of the actual crash rate.

Legal requirements for self-driving cars further complicate matters. In California (arguably the jurisdiction covering most automated vehicles), every crash involving a self-driving car, regardless of how minor, must be reported. Thus, we have a situation in which we are attempting to analyze self-driving car data, which has a full record of all crashes, relative to the current vehicle fleet, which has an incomplete record of crashes. The comparison is, as the old saying goes, apples to oranges.

The research in this report, “Automated Vehicle Crash Rate Comparison Using Naturalistic Data,” which was performed by the Virginia Tech Transportation Institute (VTTI) and commissioned by Google, sheds light on these issues. It examines both national crash data and data from naturalistic driving studies to better estimate existing crash rates, and then compares the results to data from Google’s Self-Driving Car program, which included written reports, video, and vehicle kinematic data.

This study assessed driving risk for the United States nationally and for the Google Self-Driving Car project. Driving safety on public roads was examined in three ways. The total crash rates for the Self-Driving Car and the national population were compared to (1) rates reported to the police, (2) crash rates for different types of roadways, and (3) scenarios that give rise to unreported crashes. First, crash rates from the Google Self-Driving Car project per million miles driven, broken down by severity level were calculated. The Self-Driving Car rates were compared to rates developed using national databases which draw upon police-reported crashes and rates estimated from the Second Strategic Highway Research Program (SHRP 2) Naturalistic Driving Study (NDS). Second, SHRP 2 NDS data were used to calculate crash rates for three levels of crash severity on different types of roads, broken down by the speed limit and geographic classification (termed “locality” in the study; e.g., urban road, interstate). Third, SHRP 2 NDS data were again used to describe various scenarios related to crashes with no known police report. This analysis considered whether such factors as driver distraction or impairment were involved, or whether these crashes involved rear-end collisions or road departures.

Crashes within the SHRP 2 NDS dataset were ranked according to severity for the referenced event/incident type(s) based on the magnitude of vehicle dynamics (e.g., high Delta-V or acceleration), the presumed amount of property damage (less than or greater than $1,500, airbag deployment), knowledge of human injuries (often unknown in this dataset), and the level of risk posed to the drivers and other road users (Antin, et al., 2015; Table 1). Google Self-Driving Car crashes were also analyzed using the methods developed for the SHRP 2 NDS in order to determine crash severity levels and fault (using these methods, none of the vehicles operating in autonomous mode were deemed at fault in crashes).

 

Table 1. SHRP 2 NDS Crash Severity Classifications

SHRP 2 NDS Crash Severity Level	SHRP 2 NDS Classifications
Level 1	Crashes with airbag deployment, injury, rollover, a high Delta-V, or that require towing. Injury, if present, should be sufficient to require a doctor’s visit, including those self-reported and those from apparent video. A high Delta-V is defined as a change in speed of the subject vehicle in any direction during impact greater than 20 mph (excluding curb strikes) or acceleration on any axis greater than ±2 g (excluding curb strikes).
Level 2	Crashes that do not meet the requirements for a Level 1 crash. Includes sufficient property damage that one would anticipate is reported to authorities (minimum of $1,500 worth of damage, as estimated from video). Also includes crashes that reach an acceleration on any axis greater than ±1.3 g (excluding curb strikes). Most large animal strikes and sign strikes are considered Level 2.
Level 3	Crashes involving physical conflict with another object (but with minimal damage) that do not meet the requirements for a Level 1 or Level 2 crash. Includes most road departures (unless criteria for a more severe crash are met), small animal strikes, all curb and tire strikes potentially in conflict with oncoming traffic, and other curb strikes with an increased risk element (e.g., would have resulted in a worse crash had the curb not been there, usually related to some kind of driver behavior or state, for example, hitting a guardrail at low speeds).
Level 4	Tire strike only with little or no risk element (e.g., clipping a curb during a tight turn). Distraction may or may not also be present. Note, the distinction between Level 3 and Level 4 crashes is that Level 3 crashes would have resulted in a worse crash had the curb not been there while Level 4 crashes would not have due to the limited risk involved with the curb strike. Level 4 crashes are considered to be of such minimal risk that most drivers would not consider these incidents to be crashes; therefore, they have been excluded from this analysis.

 

When compared to national crash rate estimates that control for unreported crashes (4.2 per million miles), the crash rates for the Self-Driving Car operating in autonomous mode when adjusted for crash severity (3.2 per million miles; Level 1 and Level 2 crashes) are lower. These findings reverse an initial assumption that the national crash rate (1.9 per million miles) would be lower than the Self-Driving Car crash rate in autonomous mode (8.7 per million miles) as they do not control for severity of crash or reporting requirements. Additionally, the observed crash rates in the SHRP 2 NDS, at all levels of severity, were higher than the Self-Driving Car rates. Estimated crash rates from SHRP 2 (age-adjusted) and Self-Driving Car are displayed in Figure 1.

 

Figure 1. SHRP 2 NDS and Self-Driving Car Crash Rates per Million Miles 

Low exposure for self-driving vehicles (about 1.3 million miles in this study) increases the uncertainty in Self-Driving Car crash rates compared to the SHRP 2 NDS (over 34 million miles) and nearly 3 trillion vehicle miles driven nationally in 2013 (2,965,600,000,000).

As self-driving cars continue to be tested and increase their exposure, the uncertainty in their event rates will decrease. Current data suggest that self-driving cars may have low rates of more-severe crashes (Level 1 and Level 2 crashes) when compared to national rates or to rates from naturalistic data sets, but there is currently too much uncertainty in self-driving rates to draw this conclusion with strong confidence. However, the data also suggest that less-severe events (i.e., Level 3 crashes) may happen at a significantly lower rate for self-driving cars than in naturalistic settings. Additionally, when the Self-Driving Car events were analyzed using methods developed for SHRP 2, none of the vehicles operating in autonomous mode were deemed at fault. This fact, together with the reduced crash rate for less-severe events (Level 3 crashes), represents a powerful finding. This is particularly appropriate to vehicles intended for lower-speed use where less-severe events are the most likely to be encountered by the newer generation of the Self-Driving Car fleet.

Read the full final report here.

---