• Flexible 24/7 access to data and test resources around the globe
• Hybrid cloud approach to minimize cloud costs
• Scalable replay using high-performance computing and storage solutions in a data center
• High-speed data connections to the global data lake  

Task

Putting autonomous vehicles on the road requires an enormous effort. To accomplish this task, developers often work in international teams across the globe. This also applies to the issue of validation. The developers must therefore be able to exchange large amounts of data with each other via the cloud and efficiently share global test resources.  
 

Challenge 

In many cases nowadays, data replay systems, just like other test systems, are still located at customer locations, and are often isolated or have only limited access to the cloud. This makes it difficult to leverage validation resources from globally distributed development teams and scale the test capacities required by the development of autonomous vehicles. As a result, there is an increasing trend among automotive manufacturers and suppliers to use data centers with their test equipment, where they can find flexible storage and computing capacities as well as fast connections to their cloud providers. At the same time, however, such an approach raises concerns about the security and cost of cloud use.

Solution

Due to the technology-agnostic approach that dSPACE takes in the development of its solutions, the dSPACE data replay systems are ideally prepared for use in the hybrid cloud, i.e., in data centers with a high-performance connection to the public cloud. Through this combination, the large flood of sensor data collected during a logging campaign can first be transmitted to a data center and examined and preprocessed there, instead of uploading all sensor data directly to the cloud and processing it there. Thanks to IVS (the dSPACE sensor management platform) deployed in a hybrid cloud, significant savings in cloud costs can be achieved. IVS makes it possible to generate so-called previews from sensor data, which require only a fraction of the storage space of original data. Once uploaded to the cloud environment, users worldwide can work with these previews in IVS, efficiently search-in, filter, and select data sets, as if they were directly working on the original data. On the other hand, all computationally intensive processes such as tagging and indexing can still be performed on corresponding original data in the data center to lower cost compared with doing that in the cloud. With IVS, high computing and storage capacities and the virtualization approach in the data center can be used optimally. After gaining data insights in a time-intensive preprocessing, redundant and irrelevant sensor data can be easily filtered out and only the valuable recordings are uploaded to the cloud and kept there permanently for later annotation and data replay.
Further savings potential arises from the flexible use of the data center infrastructure and the scalability of the replay. This allows a large number of systems to be configured, operated in parallel in 24/7 mode, and monitored remotely. The Replay Server in the data center is often operated in a virtualized Docker-based environment. This means that data replay systems can be flexibly switched on and off and orchestrated as required. The same flexibility applies to the provision of test scenarios and data. Data exchange with the public cloud and the data center takes place via a broadband connection of up to 100 Gbit/s per connection, which allows direct streaming of data to several data replay systems at the same time. To decrease fluctuations in data provision and to further optimize cloud egress costs, it is also possible to temporarily store the data in the data center before use. For test management and orchestration of data replay tests, dSPACE offers the cloud-based tool SIMPHERA, which uses flexible Docker technology and is operated in the Kubernetes cluster. In conjunction with the data center infrastructure, this allows test execution to be optimally scaled. dSPACE cooperates with market leaders such as AWS (Amazon Web Services) and Equinix on the topic of hybrid cloud in the field of public cloud and data centers.