Operational Safety Assessment Metrics for ADS-Equipped Vehicles
What does it mean for an automated vehicle to drive safely? How does it compare to human-driven vehicles? In this multi-phase, OSA metrics project, IAM aims to define a concise set of metrics to inform an AV operational safety performance assessment. Through IAM’s Test Network, the team can test the metrics in both controlled and real-world intersections and locations.
Phase I: August 2019 - April 2020
The first phase included the design and preliminary evaluation of OSA metric definitions, the development of a camera and AI-based, off-board detection and tracking algorithm, as well as preliminary evaluation of various possibilities for an off-board data capture system. The OSA metrics are a mix of existing, adapted, and novel metrics, and the foundational OSA (previously DSPA) metrics paper was published (see Wishart et al., 2020, below). The project is supported by IAM’s test network site operated by the Maricopa County Department of Transportation in Anthem, Arizona.
This compilation of footage from the IAM Test Network site in Anthem, Arizona features how the IAM team uses MCDOT intersection cameras to test and measure its safety assessment metrics.
IAM published driving safety performance metrics and methods identified in this project’s first phase in the SAE International Journal.
Phase II: May 2020 - April 2021
In the second phase, the simulation and real-world data were used to evaluate the OSA metrics. Work continued on the detection and tracking algorithm defined in Phase I. Research into how to bring the quantify severity of a metrics violation as well as bringing the OSA metrics into an overall assessment methodology was initiated.
Phase III: May 2021 - April 2022
In the third phase, the evaluation of OSA metrics, specifically the safety envelope-type of metrics, in simulation and with real-world data, continued. A LIDAR-based detection and tracking algorithm was developed to compare and contrast with the camera-based algorithm. The OSA Methodology, an assessment methodology, was introduced. Research into the impact of measurement uncertainty on OSA metrics implementation was initiated. A survey of intersections and locations around the state that would serve as useful additions to the IAM Test Network, including locations with frequent collisions and existing AV deployment was also initiated.
Phase IV: May 2022 - April 2023
In the fourth phase, the OSA metrics are being aligned with those in the upcoming J3237 Recommended Practice from SAE International. A fusion of camera and LIDAR data for a detection and tracking algorithm will be developed. The OSA Methodology will be developed further, including the fidelity of the testing method and complexity and relevance of a given scenario. Locations for data collection interest will be identified and data will be collected and added to the IAM database in CyVerse.
Project Experts
