The validation of functions for automated driving cannot do without virtual driving scenarios for testing. Automation improves efficient scenario creation and enhances test coverage. This is what engineers at Stellantis have successfully implemented.
Safety has always been a top priority in the automotive industry. The safety of the intended functionality is a key requirement in the software development for automated driving. As a result, verifying new functions often takes longer than developing them. Engineers are constantly striving to improve test coverage while reducing test effort. A major challenge in validating functions for advanced driver assistances systems (ADAS) and automated driving (AD) is deriving an appropriate selection of test scenarios – in terms of quality and quantity – that sufficiently covers the test space within the operational design domain (ODD).
Operational Design Domain (ODD)
An ODD definition describes specific operating conditions in which the automated driving system is designed to properly operate. It specifies what operating parameters the automated vehicle (AV) must be able to manage; for example, weather conditions, infrastructure, location, time of day, and everything else that can have an impact on the driving situation. The ODD is thus an important part of the safety concept of a vehicle and must be valid throughout its entire service life for a particular configuration of the AV.
Design of Experiment (DOE)
With DOE, it is possible to examine almost any number of factors and their effect on several target variables. Furthermore, DOE also offers the possibility to specifically investigate the mostly underestimated interactions between variables.
Systematic Approach to ADAS Validation
“We were looking for an automated approach to systematically create the appropriate scenarios,” reports Arianna Vitulano from the Virtual Test Properties Engineering (VTPE) team of Stellantis. Daniel Li from Stellantis adds, “Our goal was a design of experiment (DOE)-based solution to create scenarios in a systematic, cost- and performance-efficient way”. The validation team worked with dSPACE to develop a tool and process based on the simulation tool suite ASM (Automotive Simulation Models). It uses a DOE test matrix for scenario specification and generates OpenDRIVE and OpenSCENARIO files as the basis for running simulations in ASM. Building a simulation-centric solution around ASM enables seamless validation across model-, software-, and hardware-in-the-loop platforms. The DOE test matrix contains highlevel specifications of traffic scenarios. It can be considered a list of abstract scenarios, which means that each test (scenario) is defined without specific details of the dynamic motion of the objects (target vehicles). It is the basis for creating logical scenarios, which contain elementary motion details for the dynamic objects and can therefore be adopted by scenario-based simulators, such as the dSPACE ASM-based test platforms.
The ADAS Automation Tool
“To improve user experience, we developed the ADAS Automation Tool with a specific graphical user interface and algorithms, which makes it possible to automate scenario creation,” concludes Giancarlo Di Mare. Stellantis-specific target motion rules allow all tests in the DOE array to be concretized in a systematic way. The ADAS Automation Tool not only satisfies these rules but also provides post-processing of the concrete scenarios to automatically avoid collisions, maintain trajectories, and finally optimize the scenarios accordingly. The same tool is then used to run the generated and consolidated scenarios and provide the results (pass/fail). It also meets the general requirement of ensuring repeatability and effectiveness of the test scenarios.
Benefiting from Flexible, Repeatable Simulation
“The open interfaces of ASM enable us to apply our test matrix to the simulation in an automated way,” Kishore Vakati describes some of the benefits. The simulation itself uses validated models from ASM Traffic and ASM Vehicle Dynamics. They enable the execution of complex traffic scenarios with extensive surrounding traffic, vulnerable road users, traffic signs, relevant roadside structures, and sensor-equipped vehicles in urban areas as well as on highways. “These flexible simulation options are a prerequisite for thorough testing of ADAS functions,” explains Yibing Shi. With ASM, the engineers of Stellantis can simulate traffic scenarios that are not safe to perform on real vehicles. The implemented automation allows observing the system behavior under varying scenario parameters. Since the simulation experiments are easy to repeat, functional improvements can be verified in regression tests. With ASM, engineers of Stellantis were able to generate very realistic traffic simulations. Roberto Rabbeni summarizes “ASM reliably supports important validation steps in ADAS development.”
Next Steps and Further Outlook
Stellantis is currently investigating OpenSCENARIO 1.1 and 2.0 to ensure compatibility with ASAM-compatible tools (e.g., the dSPACE tool chain). After the successful introduction of the ADAS Automation Tool, a roll-out to further model-in-the-loop (MIL), software-in-the-loop (SIL), and hardware-in-the-loop (HIL) platforms is ongoing. In parallel, the correlation of the scenario simulation with the real traffic environment and vehicle testing is constantly monitored to prove reliability. This way, the foundation is laid for reducing physical resources by shifting more validations to virtual test drives.
The ASM simulation tool suite plays a key role in validating functions for automated driving at Stellantis.
The article has been created in close collaboration with
Roberto Rabbeni
Roberto Rabbeni is responsible for HIL/DIL Systems & Innovation in the VTPE Team at Stellantis.
Yibing Shi
Yibing Shi is with ADAS HIL testing and validation team at Stellantis.
Kishore Vakati
Kishore Vakati is responsible for Automation & Algorithm development in the VTPE Team at Stellantis.
Giancarlo Di Mare
Giancarlo Di Mare is responsible of Virtual Test Properties Engineering (VTPE) at Stellantis.
Daniel Li
Daniel Li is responsible for ADAS virtual validation in the Functional and Safety Validation team at Stellantis.
Arianna Vitulano
Arianna Vitulano is responsible for xIL Development and Automation in the VTPE Team at Stellantis.