For the validation of driver assistance systems, Toyota relies on a process that uses dSPACE tools to create a seamless link between the real world and virtual test worlds. This allows precise, reproducible test scenarios to be carried out under controlled conditions – a decisive step toward making automated vehicles even safer and more reliable.
The development of intelligent advanced driver assistance systems (ADAS) is a key factor for Toyota on the road to the mobility of the future. Safety and reliability are top priorities, because only safe vehicles strengthen confidence in the technology and ultimately in the brand. Seamless validation is the means to achieve this goal.
But the challenge is complex: Not only known risks, but also unforeseeable dangers arising from the interaction of the vehicle with its environment must be taken into account. This is precisely where the SOTIF (Safety of the Intended Functionality, ISO 21448) approach comes in, that inspired Toyota during validation.
How Do You Test the Unexpected?
A central component of SOTIF and similar validation approaches is verification and validation under realistic and critical driving conditions. But what scenarios are crucial? How can a wide range of traffic situations be covered and the most critical scenarios captured in a targeted way? Every driving scenario is influenced by numerous factors, including:
- Road conditions (dry, wet, icy)
- Weather conditions (sunlight, rain, fog)
- Driving dynamics (acceleration, steering angle, lane changes)
- Interaction with other road users (overtaking maneuvers, distances, speeds)
Example: An overtaking maneuver on a wet road when the sun is low is normally unproblematic, but a certain combination of parameters (sunlight, acceleration, steering angle) can unexpectedly lead to an accident.
To identify such scenarios, Toyota relies on comprehensive simulations with varying parameters. The targeted combination of real data, simulation models, and test drives enables these critical situations to be identified and validated at an early stage. After all, the future of driver assistance systems lies not only in powerful software, but also in intelligent and comprehensive validation that takes account of unforeseeable risks.
From the Road to Virtual Reality: How Toyota Generates Realistic Test Scenarios
To generate such realistic test scenarios, Toyota has developed a three-stage process that converts real traffic data into high-precision simulations and thus creates a reliable basis for the validation of automated driving systems (figure 1).
First, extensive measurements were carried out on a 40-kilometer stretch of highway. Several sensors, including a lidar unit, four cameras, and a global navigation satellite system/inertial measurement unit (GNSS/IMU) system, precisely captured the road structure and traffic flow. To ensure maximum accuracy, all sensors were carefully calibrated (see also ‘Precise calibration is essential’). "The precise capture of this sensor data is essential for the correct identification of traffic objects and driving maneuvers. The sensors were therefore calibrated with high precision in order to accurately capture both the road structure and the traffic flows," explains Daiki Miyata from the Toyota Motor Corporation. The data was recorded using the AUTERA system from dSPACE. With AUTERA, the raw data from lidar and camera sensors is recorded synchronously and time-stamped. Relevant data from the vehicle bus such as acceleration, steering angle, GPS trajectory/GNSS data, etc., supplement the recording.
After data capture, the information was processed in several steps. In order to meet the data protection requirements of the GDPR, license plates were anonymized, for example. AI-supported tools from understand.ai then analyzed the data and labeled lanes, height profiles, and vehicle movements in a highly automated process. By means of continuous optimization, the detection accuracy improved steadily, resulting in an increasingly detailed picture of the captured traffic situations.
The collected data was then transferred to virtual test environments using the Traffic Virtualizer from dSPACE (figure 2). The tool allows scenarios to be generated in two ways:
- In the simplest case, recorded object lists (merged sensor data in which individual objects have been identified by the ADAS perception algorithm) are sufficient.
- Higher-quality generation is achieved by processing raw sensor data with the help of understand.ai's highly automated labeling service, which in turn provides object lists.
The tool identifies relevant traffic situations in the data using predefined abstract scenarios. The scenario parameters captured in this process can be used to create statistical evaluations such as speed distributions. This allows simulation-based tests to be controlled in a targeted way and supplied with suitable data. Simulation-capable scenarios are available at the end of the process. This enables both an exact simulation of real traffic situations (replica scenarios) as well as variation through parameterization, such as speed or vehicle distances, for example. This allows numerous changeable test cases to be created by adjusting individual or several parameters so that a slightly different driving situation results each time (see also ‘Logical scenario generation’).
The scenarios generated in this way are available in the OpenDRIVE and OpenSCENARIO formats, which enables integration into various simulation platforms.
Figure 1: The entire process is shown, from data capture to scenario generation and simulation. This end-to-end implementation ensures that realistic and repeatable tests and comprehensive validation can be carried out taking unknown risks into account.
Figure 2: Logical scenario generation with Traffic Virtualizer: The tool recognizes and generates scenarios from captured traffic data.
Real Vehicles in a Virtual Test World
After generating the test scenarios, the Toyota engineers imported them into the ADAS real car simulator (ADAS RCS), which combines real vehicles with virtual environments to create a test platform that simulates realistic driving situations without risk. A vehicle prototype, e.g., on a chassis dynamometer, is coupled directly with a virtual test environment (figure 3). The sensors, e.g., cameras, are stimulated by content on screens. This enables the precise simulation of a wide range of traffic scenarios.
"A key benefit of this virtual test environment is the ability to simulate dangerous traffic situations without risk," explains Daiki Miyata from Toyota. To test the reliability of the assistance systems, scenarios were deliberately integrated in which vehicles could collide. While such a collision would have serious consequences in the real world, there is no risk in the simulation – a decisive step toward the efficient development and reliable validation of ADAS technologies.
The ability to simulate weather and road conditions that are difficult or impossible to reproduce on real test tracks is particularly valuable.
Future Prospects: More Simulation for More Comprehensive Tests
The targeted generation of critical scenarios allows potentially unsafe situations to be identified at an early stage – precisely those that are classified as safety-critical, e.g., according to SOTIF (Safety of the Intended Functionality). By specifically testing automated vehicles with these scenarios, developers can identify weak points and minimize the safety risk for other road users. This will help increase confidence in autonomous systems and accelerate their introduction on public roads. In the further development of X-in-the-loop simulations and real test benches, Toyota is focusing on the expansion of test environments and the automation of test processes. The planned measures include the integration of advanced tools such as AURELION from dSPACE, which will make it possible to simulate even more detailed and realistic scenarios by providing sensor-realistic data to test perception algorithms. In addition, the scenario libraries are being expanded to realistically depict other rare and complex traffic situations.
With the support of dSPACE, Toyota's development department has succeeded in generating complex scenarios from real traffic flows and successfully importing them into virtual test environments with this project. The entire workflow from data capture to scenario generation was efficiently designed. The most prominent advantages included:
- Data capture with AUTERA: Real traffic flows were recorded precisely.
- Highly automatic annotation: Road details and vehicles were labeled with the help of understand.ai tools.
- Automatic scenario generation: The Traffic Virtualizer generated replica and logical scenarios based on the collected data.
- Import into ADAS RCS: Scenarios were seamlessly integrated into the virtual environment using OpenX standards such as OpenDRIVE and OpenSCENARIO.
"By integrating real data into virtual test environments, development and test times will be significantly reduced and the quality and safety of ADAS systems will be significantly improved, taking the validation of modern Toyota models to a new level," says Daiki Miyata.
With the continuous further development of these technologies, Toyota will continue to meet the increasing demands on vehicle development in the future and contribute to the design of safe and sustainable mobility. These approaches show how dSPACE products can contribute to the efficient development of modern driver assistance systems.
Figure 5: The combination of replica scenarios and logical scenarios supports a wide range of use cases and helps to increase the depth of testing.
Courtesy of Toyota Motor Corporation
This article was written in close cooperation with Daiki Miyata from Toyota Motor Corporation (TMC).
dSPACE MAGAZINE, PUBLISHED JULY 2025
Daiki Miyata
Daiki Miyata works in the Vehicle Technology Development Department and XILS Development Innovation Department at Toyota Motor Corporation, Toyota City, Japan. His main role is to build the X-in-the-loop (XIL) testing environment (ADAS RCS, etc).
Precise calibration is essential
Precise measurement data is essential in order to reproduce the real world with sufficient accuracy. The sensors on the data-logging ego vehicle must therefore be precisely coordinated, both spatially (calibration) and temporally (synchronization). Despite careful calibration, however, system-related inaccuracies remain that cannot be eliminated by improved calibration alone. Therefore, methods such as simultaneous localization and mapping (SLAM) and ego-motion compensation are used to generate spatially and temporally consistent scenes even in the case of unavoidable measurement inaccuracies.
Logical scenario generation
Toyota's aim was to automatically identify relevant sections within the recording of the 40-km test drive and to generate individual logical simulation scenarios from this. Toyota defined 10 specific driving situations that were of interest for their tests. To meet this requirement, dSPACE enhanced its tool to include functions that enabled Toyota developers to define their own behavior patterns and recognize them in the real data. For example, a cut-in maneuver can be described by a combination of simple actions and conditions such as lane keeping, lane changing, or acceleration. As a result, the software is now more flexible and expandable, so that additional behavior patterns can be added in the future with minimal development effort (figure 5).
Simulation – even in the cloud
In principle, the scenarios can be simulated in the dSPACE models ASM Traffic and AURELION. Two simulation options are available for this: The scenarios can either be executed on a hardware-in-the-loop (HIL) simulator with SCALEXIO or scaled in the cloud with SIMPHERA. SIMPHERA also enables the automated implementation of numerous parameter variations in the cloud. This means that hundreds of virtual scenarios can be derived from a single real scenario. Targeted parameter variations can also be used to make scenarios more critical, e.g., by increasing speeds or reducing distances in order to simulate critical corner cases.
ISO 26262 and ISO 21448
ISO 26262 addresses the functional safety of E/E systems in road vehicles and provides a set of rules to help prevent undue risk from system behavior – including systematic faults and random hardware failures. A functional restriction of the driving function with regard to real use cases is not taken into account in the context of ISO 26262. For example, the sensor of an assistance system might function perfectly from a technical point of view, but the assistance system might not be able to correctly recognize an object in a scenario. ISO 21448 (SOTIF) takes these aspects into account in addition to ISO 26262. According to SOTIF, scenarios are divided into known and unknown and into safe and unsafe areas. The known risks are evaluated and addressed through targeted measures and tests. Risks that have not yet been fully identified require extensive analyses and tests to detect them. The systematic analysis of the traffic data from the recorded test drives with the Traffic Virtualizer can help identify scenario categories that have not yet been taken into account in virtual tests. Potentially dangerous scenarios that are identified are then systematically managed.
At a Glance
Task:
- As part of the validation of ADAS, test scenarios are generated from measurement data recorded during real journeys.
Challenge:
- Comprehensive validation of ADAS with scenarios that cover unforeseeable hazards.
Solution:
- Synchronous, time-stamped data recording from relevant sensors with AUTERA
- Semi-automatic annotation of sensor data with tools from understand.ai (UAI)
- Automatic generation of logical scenarios with Traffic Virtualizer
Benefits:
- Development times and test times are significantly reduced
- The quality and safety of ADAS systems is significantly improved
