- Proof of correctness and accuracy of data replay
- Detailed autogenerated reporting
- Easy integration into existing test infrastructure
Task
Data replay systems are becoming increasingly larger and more flexible. They are often integrated into the infrastructure of data centers. A data replay system must ensure that a large number of parallel data streams of different types are fed into a device under test (DUT) with exact time correlation in order to create the correct precondition for the test. The originally recorded sensor data must also be adapted to the ADAS/AD ECU, which is frequently updated during further development. This increases the complexity and error-proneness of the replay. In order to ensure the accuracy and correctness of the test execution and the usability of the test results, the test departments are strongly advised to plan ongoing validation of the data replay systems used firmly into their test strategy.
Challenge
In order to demonstrate the proper functioning of a data replay system, a comparison must be made between the original sensor data used for the replay (reference) and the replay output data fed into the DUT. The evaluation is carried out both for the correctness of the content of the data and with regard to its temporal course in order to check the accuracy of the reprocessing. The difficulty lies in the fact that the correct starting point for the data comparison and the permitted deviations in terms of content and time depend on many factors and must be taken into account when comparing data. For example, especially in the case of service-oriented data transmission via SOME/IP, it must be clear in advance which messages are to be filtered out, or how the triggers for the start or completion of a replay sequence are to be detected in the input data. Furthermore, the vehicle electrical system and the DUT itself change in the course of development, and thus adjustments to data material regarding network parameters, end-to-end security, etc., must take place during the replay and must be taken into account when comparing with original data. Finally, the clocks of the replay system and the loggers used to record the replay output data do not necessarily run synchronously. This can lead to deviations in the data to be compared.
Solution
With the data replay validation package, dSPACE can provide an individual solution for the qualitative evaluation of data replay with a focus on bus data feed-in (CAN, Ethernet UDP, and SOME/IP) on request. The solution is available for RTMaps on Windows or Ubuntu Linux. The evaluation is carried out by comparing the reference data used for the replay and the output data of the replay system recorded by a logger (probe). As a result of the automatic comparison, a machine-readable or human-readable report can be generated, which includes relevant performance characteristics of the replay such as jitter, number of missing and additionally available bus frames, order of the data, statistical statement about correct and incorrect payload, etc., including graphical representation. Through a variety of setting options such as replay trigger conditions, message configuration, and filter parameters, or PTP synchronization data, the data comparison can be precisely adjusted to the respective use case.
