Mori, Ken T. ; Liang, Xu ; Elster, Lukas ; Peters, Steven (2022)
The Inadequacy of Discrete Scenarios in Assessing Deep Neural Networks.
In: IEEE Access, 10
doi: 10.1109/ACCESS.2022.3220904
Artikel, Bibliographie
Kurzbeschreibung (Abstract)
Many recent approaches for automated driving (AD) functions currently include components relying on deep neural networks (DNNs). One approach in order to test AD functions is the scenario-based approach. This work formalizes and evaluates the parameter discretization process required in order to yield concrete scenarios for which an AD function can be tested. Using a common perception algorithm for camera images, a simulation case study is conducted for a simple static scenario containing one other vehicle. The results are analyzed with methods akin to those applied in the domain of computational fluid dynamics (CFD). The performance of the perception algorithm shows strong fluctuations even for small input changes and displays unpredictable outliers even at very small discretization steps. The convergence criteria as known from CFD fail, meaning that no parametrization is found which is sufficient for the validation of the perception component. Indeed, the results do not indicate consistent improvement with a finer discretization. These results agree well with theoretical attributes known for existing neural networks. However, the impact appears to be large even for the most basic scenario without malicious input. This indicates the necessity of directing more attention towards the parameter discretization process of the scenario-based testing approach to enable the safety argumentation of AD functions.
| Typ des Eintrags: | Artikel |
|---|---|
| Erschienen: | 2022 |
| Autor(en): | Mori, Ken T. ; Liang, Xu ; Elster, Lukas ; Peters, Steven |
| Art des Eintrags: | Bibliographie |
| Titel: | The Inadequacy of Discrete Scenarios in Assessing Deep Neural Networks |
| Sprache: | Englisch |
| Publikationsjahr: | 2022 |
| Verlag: | IEEE |
| Titel der Zeitschrift, Zeitung oder Schriftenreihe: | IEEE Access |
| Jahrgang/Volume einer Zeitschrift: | 10 |
| DOI: | 10.1109/ACCESS.2022.3220904 |
| URL / URN: | https://ieeexplore.ieee.org/document/9943537 |
| Kurzbeschreibung (Abstract): | Many recent approaches for automated driving (AD) functions currently include components relying on deep neural networks (DNNs). One approach in order to test AD functions is the scenario-based approach. This work formalizes and evaluates the parameter discretization process required in order to yield concrete scenarios for which an AD function can be tested. Using a common perception algorithm for camera images, a simulation case study is conducted for a simple static scenario containing one other vehicle. The results are analyzed with methods akin to those applied in the domain of computational fluid dynamics (CFD). The performance of the perception algorithm shows strong fluctuations even for small input changes and displays unpredictable outliers even at very small discretization steps. The convergence criteria as known from CFD fail, meaning that no parametrization is found which is sufficient for the validation of the perception component. Indeed, the results do not indicate consistent improvement with a finer discretization. These results agree well with theoretical attributes known for existing neural networks. However, the impact appears to be large even for the most basic scenario without malicious input. This indicates the necessity of directing more attention towards the parameter discretization process of the scenario-based testing approach to enable the safety argumentation of AD functions. |
| Fachbereich(e)/-gebiet(e): | 16 Fachbereich Maschinenbau 16 Fachbereich Maschinenbau > Fachgebiet Fahrzeugtechnik (FZD) |
| Hinterlegungsdatum: | 20 Dez 2022 13:56 |
| Letzte Änderung: | 28 Dez 2022 09:49 |
| PPN: | 503202789 |
| Export: | |
| Suche nach Titel in: | TUfind oder in Google |
 |
Frage zum Eintrag |
Optionen (nur für Redakteure)
 |
Redaktionelle Details anzeigen |
Drucken
Impressum