TU Darmstadt / ULB / TUbiblio

Towards a Generally Accepted Validation Methodology for Sensor Models - Challenges, Metrics, and First Results

Rosenberger, Philipp and Wendler, Jan Timo and Holder, Martin Friedrich and Linnhoff, Clemens and Berghöfer, Moritz and Winner, Hermann and Maurer, Markus (2019):
Towards a Generally Accepted Validation Methodology for Sensor Models - Challenges, Metrics, and First Results.
In: Grazer Symposium Virtuelles Fahrzeug, Graz, Austria, 07.-08.05.2019, [Online-Edition: https://tuprints.ulb.tu-darmstadt.de/8653],
[Conference or Workshop Item]

Abstract

In order to significantly reduce the testing effort of autonomous vehicles, simulation-based testing in combination with a scenario-based approach is a major part of the overall test concept. But, for sophisticated simulations, all applied models have to be validated beforehand, which is the focus of this paper. The presented validation methodology for sensor system simulation is based on a state-of-the-art analysis and the derived necessary improvements. The lack of experience in formulating requirements and providing adequate metrics for their usage in sensor model validation, in contrast to e.g. vehicle dynamics simulation, is addressed. Additionally, the importance of valid measurement and reference data is pointed out and especially the challenges of repeatability and reproducibility of trajectories and measurements of perception sensors in dynamic multi-object scenarios are shown. The process to find relevant scenarios and the resulting parameter space to be examined is described. At the example of lidar point clouds, the derivation of metrics with respect to the requirements is explained and exemplary evaluation results are summarized. Based on this, extensions to the state-of-the-art model validation method are provided.

Item Type: Conference or Workshop Item
Erschienen: 2019
Creators: Rosenberger, Philipp and Wendler, Jan Timo and Holder, Martin Friedrich and Linnhoff, Clemens and Berghöfer, Moritz and Winner, Hermann and Maurer, Markus
Title: Towards a Generally Accepted Validation Methodology for Sensor Models - Challenges, Metrics, and First Results
Language: German
Abstract:

In order to significantly reduce the testing effort of autonomous vehicles, simulation-based testing in combination with a scenario-based approach is a major part of the overall test concept. But, for sophisticated simulations, all applied models have to be validated beforehand, which is the focus of this paper. The presented validation methodology for sensor system simulation is based on a state-of-the-art analysis and the derived necessary improvements. The lack of experience in formulating requirements and providing adequate metrics for their usage in sensor model validation, in contrast to e.g. vehicle dynamics simulation, is addressed. Additionally, the importance of valid measurement and reference data is pointed out and especially the challenges of repeatability and reproducibility of trajectories and measurements of perception sensors in dynamic multi-object scenarios are shown. The process to find relevant scenarios and the resulting parameter space to be examined is described. At the example of lidar point clouds, the derivation of metrics with respect to the requirements is explained and exemplary evaluation results are summarized. Based on this, extensions to the state-of-the-art model validation method are provided.

Divisions: 16 Department of Mechanical Engineering
16 Department of Mechanical Engineering > Institute of Automotive Engineering (FZD)
16 Department of Mechanical Engineering > Institute of Automotive Engineering (FZD) > Driver Assistance
16 Department of Mechanical Engineering > Institute of Automotive Engineering (FZD) > Safety
16 Department of Mechanical Engineering > Institute of Automotive Engineering (FZD) > Test Methods
Event Title: Grazer Symposium Virtuelles Fahrzeug
Event Location: Graz, Austria
Event Dates: 07.-08.05.2019
Date Deposited: 09 Jun 2019 19:55
Official URL: https://tuprints.ulb.tu-darmstadt.de/8653
URN: urn:nbn:de:tuda-tuprints-86536
Export:

Optionen (nur für Redakteure)

View Item View Item