The development of autonomous vehicles leads to immense test and validation efforts. To handle complexity and costs, SERES has opted for a testing system from dSPACE that can be used early and flexibly in the development process – with real sensors in the loop.
SERES is a global transportation technology company that develops and manufactures intelligent electric vehicles to create safer, cleaner, and more sustainable communities. We at SERES technologies are focused on delivering improved safety, convenience, and performance to a global market. Our company owns and operates manufacturing, assembly, and research and development facilities across the U.S., China, and Japan.
This is an article from dSPACE Magazine 01/2020. You can read the complete edition online here.
The complex AD solution that is currently under development has to be reliably validated before it hits the road.
SilkRides is a business unit of SERES that develops autonomous driving (AD) technology. Our team has an OEM heritage with Silicon Valley roots to provide a cost-competitive and open autonomy solution for carmakers. Established in 2017, SilkRides has demonstrated Level 3 and Level 4 autonomous driving in urban and highway scenarios.
The team is set to cover the complete technology required to develop and manufacture autonomous vehicles. This includes expertise in perception, planning, and control – not only for development but also for the validation of these safety-related areas as well as over-the-air (OTA) software updates, hardware design, etc. A key component is the domain control unit, which is partly based on artificial intelligence. It evaluates the sensor data and determines the driving strategy. In a first step, it is implemented as a prototype controller, which is gradually developed further for mass production.
Verification and validation are carried out in consecutive steps, which are repeated according to test requirements.
To prove that an autonomous driving system is as safe as an average human driver, billions of real-world driving miles for validation might be required. Moreover, each update of the software also has to undergo rigorous testing. Here at SERES, we have limited resources and a tight deadline to accomplish this task, and road testing on a large scale is unrealistic. To speed up our development and validation cycle, we look to establish our verification and validation pipeline with various levels of simulation, where key scenarios and corner cases can be tested for each software release before road tests.
To test the entire chain of effects, the dSPACE radar test bench provides over-the-air stimulation of the radar sensor. If desired, the simulation can include the front bumper and parts of the chassis of the vehicle. This way, all the software and hardware layers can be taken into account, from detecting the signal at the front of the radar to evaluating it in the radar ECU. This very compact test bench fundamentally consists of an anechoic chamber with integrated antennas for send/receive functionality, a calibrated dSPACE Automotive Radar Test System (DARTS), and a SCALEXIO HIL simulator. For the test, the radar sensor is locked into the chamber, where it is stimulated with realistic radar echoes. The coherent echoes allow for the radar ECU to reliably determine the distance, velocity, radar cross section (RCS), and angle of the radar objects. The front radar is tested by using two DARTS 9030-M units. For the corner radars, SERES installed a DARTS 9030-MS unit, which is particularly strong at short-range stimulation. This configuration is sufficient to test all relevant driving scenarios.
At SilkRides, each software release goes through multiple levels of testing before it reaches the test vehicle. First, unit tests and module tests are performed to ensure that the new software components behave as intended. Then the new components are integrated with the rest of the decision and planning modules for software-in-the-loop (SIL) testing, where the software pilots a simulated vehicle through relevant test cases to evaluate performance. If the SIL testing results are positive, the new software release will be compiled on the domain controller hardware for hardware-in-the-loop (HIL) testing. The process has a certain data consistency for test artifacts, such as virtual roads. During the SIL and HIL testing process, many issues invol-ving the software and the interaction between software and hardware are observed and fixed, reducing the number of test vehicles and road test mileage required to validate the software releases.
“To prove that an autonomous driving system is as safe as an average human driver, billions of real-world driving miles of validation might be required. The dSPACE test system in our lab provides an efficient way of testing real stuff on virtual roads.”
Ziqi Zhu, SERES
There are a few key requirement that have to be solved by HIL testing for the development of autonomous driving at SilkRides:
Overview of the sensor setup in the vehicle.
To perform detailed and comprehensive lab testing, we went for a flexible and complete setup that covers the SilkRides AD stack. Together with dSPACE, we defined a system that can handle integration testing of sensors, controllers, and actuators. It consists of a HIL simulator that realistically represents our vehicles. It is further extended by four synchronized sensor test benches, which bring the real radars and cameras into the loop. The ability to use real components also allows us to evaluate the performance of different components at the early stages of development.
“The radar test bench gives us the opportunity to evaluate and test radar sensors early on. This greatly improves development of the software for radar processing and autonomous driving.”
Samuel Rayseldi, SERES
We started testing by performing test drives with ASM Traffic, the traffic simulation model of the Automotive Simulation Models (ASM) tool suite. It lets us define traffic vehicles, pedestrians, traffic signs, etc. on roads imported from HD maps. Moreover, ASM Traffic supports the creation of test scenarios for the assistant functions of our vehicles, such as adaptive cruise control, lane keeping, overtaking assistants, etc. The overall test automation is handled by AutomationDesk, providing additional test options, such as signal manipulation and test reporting.
The simulated world is then fed into the sensor test benches to stimulate the radar and camera sensors. The sensor signals are the inputs for the device under test, which is our controller running the developed AD software with perception and sensor fusion algorithms. This way, all AD components are integrated and their real behavior is taken into account. Every virtual test drive can be monitored by 3-D animation in real time.
The camera box for over-the-air stimulation. The camera sensor (left) is aligned to the monitor (right).
To test the sensing properties of a camera, the dSPACE camera box provides over-the-air stimulation of the camera’s imager chip. For this purpose, it is equipped with a monitor that displays the simulated sensor environment (e.g., a traffic scenario with multiple vehicles, pedestrians, roadside structures, etc.). The camera then further processes the data. To minimize any additional light sources or glare, everything is installed in a closed chamber.
The test system, based on the HIL simulator and the test benches, gives us various test possibilities that lead to excellent coverage. They also help perform quick evaluations under identical conditions. The following evaluations and tests were performed and show what SERES has achieved:
The field of autonomous driving is quickly evolving, so investing in assets that can adapt to changes is crucial. Over-the-air radar and camera test benches from dSPACE were selected for their flexibility in adapting to different sensor types and sensor configurations. The over-the-air approach makes it particularly easy to perform plausibility tests. This way the behavior of the sensors (complete control unit including the signal processing software) is evaluated under defined conditions – and especially in borderline cases. The cases in which objects are detected by a sensor or not detected are fundamental for the development of the software that further processes the sensor signals.
SilkRides relies on automotive Tier 1 suppliers for radar sensors, so accurate benchmarking of the performance between the sensors is one of the key tasks to be considered when narrowing down the supplier. Radar performance tests on the vehicle generally require a large site, and often require the host vehicle and the target vehicle to be moving simultaneously with ground truth devices installed. With the closed-loop radar test bench, many of the tests can be conducted in the lab, providing clear and consistent re- sults. The dSPACE Automotive Radar Test System (DARTS), which is installed in every radar test bench, contributes to this. It generates radar echoes over-the-air with moving antennas so that the characteristics of a sensor can be determined exceptionally accurately.
Visualization of the imported map data from the Oakland Bay Bridge area.
Our 1x camera, 1x front radar, 2x corner radar HIL setup is enough to simulate most of the highway driving functions in our system, since the corner radars can be configured to be on the front, side, or rear of the vehicle. As next steps, additional cameras, radars, lidars, ultrasonic sensors, and GNSS simulators can be included on the SCALEXIO platform. Due to the flexible system setup, these can be either hardware or software extensions. For instance, the system supports the integration of a lidar model from the dSPACE Sensor Simulation tool chain that simulates the complete lidar transmission channel, including the 3-D environment.
With the development of autonomous driving systems of SAE Level 3 or above, many parts of the system must be fail-operational in all of the operational design domains. Such failure conditions can be difficult and sometimes dangerous to replicate in the vehicle. The hand-off between primary function and backup function in case of failure and the safe stop maneuvers in various road conditions can all be simulated in the HIL test.
dSPACE tools, such as AutomationDesk, allow us to automatically configure the test benches upon startup, which significantly reduces manual entry steps. Moreover, many of our tests can be automatically executed without user intervention, allowing a fixed set of test cases to be systematically evaluated for each new software release or hardware modification.
The SilkRides highway driving uses third-party maps to assist in lane keeping, lane change, and routing functions. The integration between the dSPACE ASM and HD map import ensures a high-fidelity simulation environment compared to real-world testing. Here in the San Francisco Bay area, the entrances and the exits of the Bay Bridge have presented many challenging scenarios. We have im- ported the Bay Bridge map into ASM and recreated many of these tricky test cases in our HIL test setup.
The dSPACE test system provides a unique way of integrating sensors and controllers in a virtual vehicle and testing them together. The use of this extra-ordinarily realistic test environment allows us to gain meaningful insights into the performance of the hardware and software components used, even in the early stages of development. This accelerates the development process because groundbreaking decisions can be made at an early stage. Since test cases can be easily reused, regression testing can be performed to reliably verify troubleshooting. New requirements can be met by expanding both the flexible test system and the test libraries. We look forward to bringing new vehicles onto the road that have been validated by a robust and reliable test system.
Ziqi Zhu, SERES (Santa Clara, CA, USA)
Hala Al-Khalil, SERES (Santa Clara, CA, USA)
Samuel Rayseldi, SERES (Santa Clara, CA, USA)