A Framework for Automated Driving System Testable …

1 DOT HS 812 623 September 2018 A Framework for Automated Driving System Testable Cases and Scenarios i DISCLAIMER This publication is distributed by the Department of Transportation, National Highway Traffic Safety Administration, in the interest of information exchange. The opinions, findings and conclusions expressed in this publication are those of the authors and not necessarily those of the Department of Transportation or the National Highway Traffic Safety Administration. The United States Government assumes no liability for its contents or use thereof. If trade or manufacturers names are mentioned, it is only because they are considered essential to the object of the publication and should not be construed as an endorsement. The United States Government does not endorse products or manufacturers.

2 Suggested APA Format Citation: Staplin, L., Mastromatto, T., Lococo, K. H., Kenneth W. Gish, K. W., & Brooks, J. O. (2018, September). The effects of medical conditions on Driving performance (Report No. DOT HS 812 623). Washington, DC: National Highway Traffic Safety Administration. i Technical Report Documentation Page 1. Report No. DOT HS 812 623 2. Government Accession No. 3. Recipient's Catalog No. 4. Title and Subtitle A Framework for Automated Driving System Testable Cases and Scenarios 5. Report Date September 2018 6. Performing Organization Code 7. Authors Eric Thorn,** Shawn Kimmel,** Michelle Chaka* 8. Performing Organization Report No. 9. Performing Organization Name and Address Virginia Tech Transportation Institute* 3500 Transportation Research Plaza (0536) Blacksburg, VA 24061; Southwest Research Institute** 6220 Culebra Rd.

3 San Antonio, TX 78238; Booz Allen Hamilton** 20 M St. SE Washington DC 20003 10. Work Unit No. (TRAIS) 11. Contract or Grant No. 12. Sponsoring Agency Name and Address National Highway Traffic Safety Administration 1200 New Jersey Avenue SE. Washington, DC 20590 13. Type of Report and Period Covered Final Report 14. Sponsoring Agency Code 15. Supplementary Notes 16. Abstract This report describes a Framework for establishing sample preliminary tests for Automated Driving Systems. The focus is on light duty vehicles exhibiting higher levels of automation, where the System is required to perform the full dynamic Driving task, including lateral and longitudinal control, as well as object and event detection and response. 17. Key Words Automated Driving systems, fail-safe mechanisms, object and event detection and response, tests, operational design domain 18.

4 Distribution Statement No restrictions. This document is available to the public through the National Technical Information Service, 19. Security Classif. (of this report) -Unclassified 20. Security Classif. (of this page) Unclassified 21. No. of Pages 180 22. Price Form DOT F (8-72) Reproduction of completed page authorized ii EXECUTIVE SUMMARY Automated Driving systems (ADS) are being developed to perform the primary functions of the dynamic Driving task (DDT). These technologies hold great promise to improve safety and mobility for transportation. The goal of this research was to develop an example of a preliminary test Framework for ADS that are in development and may come to market in the near to mid future. The following steps were conducted to support the development of the sample test Framework .

5 1. Identify concept ADS 2. Identify attributes that define the operational design domain (ODD) 3. Identify object and event detection and response (OEDR) capabilities 4. Identify and assess failure modes and failure mitigation strategies Technologies of interest in this work included light-duty Automated Driving functions that fell within Level 3 (L3) to Level 5 (L5) of the SAE1 levels of Driving automation (SAE International, 2016). The functions were identified based on prototype vehicles and conceptual systems. A literature review which included popular media, press releases, technical journals, and conference proceedings was performed. This review identified potential concept ADS being developed or proposed by original equipment manufacturers (OEMs), suppliers, technology companies, and other organizations.

6 The identified ADS were categorized into a set of generic names. T he terminology was modified to ADS features (as opposed to functions) to more closely align with the standardization community s language. Twenty-four conceptual features were identified, and although a thorough search was conducted, the list is not exhaustive. The identified features were grouped into seven generic categories. L4 Highly Automated Vehicle/Transportation Network Company (TNC) L4 Highly Automated Highway Drive L4 Highly Automated Low Speed Shuttle L4 Highly Automated Valet Parking L4 Highly Automated Emergency Takeover L3 Conditional Automated Highway Drive L3 Conditional Automated Traffic Jam Drive The generic names were developed to align with terminology from the SAE levels of Driving automation ( , conditional Driving automation [L3], high Driving automation [L4], and full Driving automation [L5]).

7 Three of these generic features were selected to further support the development of an example of a testing Framework for ADS (L3 Conditional Automated Traffic Jam Drive, L3 Conditional Automated Highway Drive, and L4 Highly Automated Vehicle/TNC). 1 In 2006 the Society of Automotive Engineers changed its name to SAE International. It s standards are still called SAE standards. iii The ODD describes the specific operating domains in which the ADS is designed to function. The ODD will likely vary for each ADS feature on a vehicle and specifies the condition in which that feature is intended and able to operate with respect to roadway types, speed range, lighting conditions, weather conditions, and other operational constraints. The ODD is specified by the technology developer, and the ADS should be able to identify whether it is operating within or outside of that ODD.

8 A literature review was performed for all the generic ADS features to identify the attributes that define the ODD. The review included popular media, press releases, technical journals, videos, and conference proceedings. An ODD taxonomy for this report was then defined. This taxonomy is hierarchical and includes the following top-level categories. Physical Infrastructure Operational Constraints Objects Connectivity Environmental Conditions Zones Some of the challenges associated with ODD elements include their variability ( , rain droplet sizes can vary greatly: light rain; moderate rain; and heavy rain), as well as identifying or defining their boundaries. The work performed to identify the ODD lays a foundational Framework from which the ODD can be further defined and delineated by the developer, and from which industry standards for ODD definition can be established.

9 OEDR refers to the subtasks of the DDT that include monitoring the Driving environment (detecting, recognizing, and classifying objects and events and preparing to respond as needed) and executing an appropriate response to such objects and events ( , as needed to complete the DDT and/or DDT fallback; (SAE International, 2016). A notional concept of operations (ConOps) was considered for the three selected ADS features. This served as a basis to perform an evaluation of the normal Driving scenarios each ADS feature may encounter, including expected hazards ( , other vehicles, pedestrians) and sporadic/fluctuating events ( , emergency vehicles, construction zones). Baseline ODDs were identified for each of the features to frame this analysis. These baseline ODDs and scenario analyses were used to identify important OEDR functional capabilities.)

10 This analysis, along with the survey of ADS features, helped to identify two key sets of behaviors for the selected ADS features. Tactical Maneuver Behaviors OEDR Behaviors Tactical maneuver behaviors may be viewed as more control-related tasks ( , lane following, turning). OEDR behaviors may be regarded as perception and decision-making related tasks iv ( , detecting and responding to pedestrians). This analysis generated a list of fundamental objects that may be relevant to an ADS s Driving task, as well as important events, which can be viewed as interactions with those objects. A list of potential responses the ADS could implement was identified, and these responses were mapped to the objects and events. To develop a preliminary testing Framework , existing test methods and tools were identified and evaluated to formulate an appropriate, comprehensive testing architecture.