Löckel, Stefan Alexander (2022)
Machine Learning for Modeling and Analyzing of Race Car Drivers.
Technische Universität Darmstadt
doi: 10.26083/tuprints-00020218
Dissertation, Erstveröffentlichung, Verlagsversion
Kurzbeschreibung (Abstract)
Motorsport teams constantly strive for engineering the fastest vehicle among all competitors in order to win races. This optimization process is a complex task, aiming to minimize the average lap time of a race car while considering several constraints. The vehicle needs to be modified for different tracks and various conditions while following the mandatory regulations of the corresponding racing series. Although vehicle dynamics are well understood and modeled, direct optimization of the vehicle's acceleration potential is only of limited use as it solely minimizes the theoretical lap time but neglects the influence of the human driver. As the driver plays a substantial role in the overall vehicle system on track, the race car needs to be set up to fit the individual driving style in order to reach maximum performance. For this reason, many motorsport teams are nowadays utilizing Human-Driver-in-the-Loop (HDiL) simulators that facilitate rapid testing of modified parts and systems at various tracks. The simulations provide direct insights into the overall system comprised of the driver, vehicle, and track while allowing to receive feedback from the human driver. Whereas this direct inclusion of the driver into the vehicle development process is beneficial for testing, the simulators are not suited for extensive lap time optimization due to long evaluation times and high resource requirements. Hence, a robust and human-like race driver model that can imitate individual driving styles in fully virtual vehicle simulations is required to effectively increase the number of tests. At the same time, however, the modeling of human race driver behavior is a highly challenging task: The vehicle is constantly maneuvered at its acceleration potential, closely approaching or even exceeding the stability limits, which ultimately requires a control policy that is robust. Furthermore, each driver shows a certain amount of variability even in a deterministic simulation environment. This variance results from intentional adaptation to optimize the driving performance and human imprecision and needs to be modeled. Despite achieving similar performances, even top-class race drivers exhibit differing individual driving styles and preferences for specific vehicle characteristics. This thesis focuses on analyzing and modeling race driver behavior to support the modern vehicle development process and to better understand the human driver. After an introduction to motorsports, related work, and essential foundations, Probabilistic Modeling of Driver Behavior (ProMoD) is presented as a novel approach for the imitation of human driver behavior in a race driving environment. In this framework, the task of driving is separated into different modules inspired by knowledge on race driver behavior and autonomous driving. Human variance is encoded with a distribution of driving lines using Probabilistic Movement Primitives (ProMPs) and clothoids are used for planning a feasible path ahead. These path planning features, combined with basic perception features, are subsequently used for supervised training of a recurrent neural network using human demonstrations. The resulting probabilistic driving policy for each driver is extensively tested and benchmarked in a simplified car racing simulation environment from OpenAI Gym. It is demonstrated that ProMoD produces human-like driver actions while being considerably more robust than direct end-to-end supervised learning and an adapted version of the Dataset Aggregation method (DAgger). The second part of this thesis focuses on the development of a structured method to analyze individual race driving styles and on the extension of ProMoD to a professional motorsport HDiL simulator, both based on data from professional race drivers. The herein proposed Driver Identification and Metric Ranking Algorithm (DIMRA) calculates an extensive set of lap-based metrics describing the driving style, mainly depending on the driver actions. To maintain interpretability and reduce complexity, DIMRA reduces the number of metrics using Sequential Backward Selection (SBS) combined with k-medoids clustering. The resulting track-depending subsets facilitate linear separability of driving styles with purities of up to 98% and demonstrate transferability from simulation to real track data. In addition, the previously proposed ProMoD framework is considerably extended and adapted to a complex motorsport driving simulator environment. Besides utilizing a new distance-based ProMP representation and validating driving line samples, the previous behavioral cloning approach is modified to prevent divergence due to the known issue of compounding errors. The resulting driver model can generate complete laps with competitive lap times while maintaining human-like behavior in actions and the utilization of driver assistance systems. Even on mirrored tracks and with considerably reduced tire grip ProMoD can approximately produce expected lap times. The third part of this thesis focuses on providing generalization and learning abilities for the introduced driver model. For this purpose, basic learning procedures and adaptation techniques of race drivers are identified from the literature and extended by an expert interview with a professional race engineer. Based on the hereby generated insights, two methods are developed to further enhance the imitation capabilities of ProMoD: By introducing an approach to estimate target trajectory distributions for unknown tracks based on experiences from other tracks, the driver model achieves track generalization capabilities and is enabled to generate close to competitive lap times on new tracks. Furthermore, an adaptation process is presented that enables ProMoD to adapt its behavior based on experiences from previous laps. By using the professional racing simulator for evaluation, it is shown that the driver model can learn from mistakes from past laps and completes previously unfinished laps with increased performance. In summary, the findings of this thesis contribute to a better understanding of the human driver and pave the way for advanced lap time optimization with consideration of individual driver characteristics. Furthermore, the findings aid enhanced modern vehicle development and potentially future autonomous racing. Besides this race driving setting, such a driver model should be particularly helpful for the development of driver assistance systems in road cars, as it can trigger human-like Traction Control (TC) interventions. Due to its modular architecture, ProMoD may be extended in various ways in future research. Additional adaptation methods may be introduced that further increase the ability of the driver model to learn from experience and modified feature definitions might yield an even more accurate imitation of the human driver. Furthermore, direct encoding of prior knowledge, as the characteristics of the vehicle setup, could further enhance the learning process. In addition, the neural network which models the internal action selection process of the human driver could be directly adapted, for instance by using reinforcement learning techniques. Ultimately, the proposed framework may be applied to comparable, challenging vehicle control tasks like drifting a car, maneuvering drones, or flying planes.
Typ des Eintrags: | Dissertation | ||||
---|---|---|---|---|---|
Erschienen: | 2022 | ||||
Autor(en): | Löckel, Stefan Alexander | ||||
Art des Eintrags: | Erstveröffentlichung | ||||
Titel: | Machine Learning for Modeling and Analyzing of Race Car Drivers | ||||
Sprache: | Englisch | ||||
Referenten: | Peters, Prof. Dr. Jan ; Gerdes, Prof. Dr. J. Christian | ||||
Publikationsjahr: | 2022 | ||||
Ort: | Darmstadt | ||||
Kollation: | xviii, 131 Seiten | ||||
Datum der mündlichen Prüfung: | 25 Oktober 2021 | ||||
DOI: | 10.26083/tuprints-00020218 | ||||
URL / URN: | https://tuprints.ulb.tu-darmstadt.de/20218 | ||||
Kurzbeschreibung (Abstract): | Motorsport teams constantly strive for engineering the fastest vehicle among all competitors in order to win races. This optimization process is a complex task, aiming to minimize the average lap time of a race car while considering several constraints. The vehicle needs to be modified for different tracks and various conditions while following the mandatory regulations of the corresponding racing series. Although vehicle dynamics are well understood and modeled, direct optimization of the vehicle's acceleration potential is only of limited use as it solely minimizes the theoretical lap time but neglects the influence of the human driver. As the driver plays a substantial role in the overall vehicle system on track, the race car needs to be set up to fit the individual driving style in order to reach maximum performance. For this reason, many motorsport teams are nowadays utilizing Human-Driver-in-the-Loop (HDiL) simulators that facilitate rapid testing of modified parts and systems at various tracks. The simulations provide direct insights into the overall system comprised of the driver, vehicle, and track while allowing to receive feedback from the human driver. Whereas this direct inclusion of the driver into the vehicle development process is beneficial for testing, the simulators are not suited for extensive lap time optimization due to long evaluation times and high resource requirements. Hence, a robust and human-like race driver model that can imitate individual driving styles in fully virtual vehicle simulations is required to effectively increase the number of tests. At the same time, however, the modeling of human race driver behavior is a highly challenging task: The vehicle is constantly maneuvered at its acceleration potential, closely approaching or even exceeding the stability limits, which ultimately requires a control policy that is robust. Furthermore, each driver shows a certain amount of variability even in a deterministic simulation environment. This variance results from intentional adaptation to optimize the driving performance and human imprecision and needs to be modeled. Despite achieving similar performances, even top-class race drivers exhibit differing individual driving styles and preferences for specific vehicle characteristics. This thesis focuses on analyzing and modeling race driver behavior to support the modern vehicle development process and to better understand the human driver. After an introduction to motorsports, related work, and essential foundations, Probabilistic Modeling of Driver Behavior (ProMoD) is presented as a novel approach for the imitation of human driver behavior in a race driving environment. In this framework, the task of driving is separated into different modules inspired by knowledge on race driver behavior and autonomous driving. Human variance is encoded with a distribution of driving lines using Probabilistic Movement Primitives (ProMPs) and clothoids are used for planning a feasible path ahead. These path planning features, combined with basic perception features, are subsequently used for supervised training of a recurrent neural network using human demonstrations. The resulting probabilistic driving policy for each driver is extensively tested and benchmarked in a simplified car racing simulation environment from OpenAI Gym. It is demonstrated that ProMoD produces human-like driver actions while being considerably more robust than direct end-to-end supervised learning and an adapted version of the Dataset Aggregation method (DAgger). The second part of this thesis focuses on the development of a structured method to analyze individual race driving styles and on the extension of ProMoD to a professional motorsport HDiL simulator, both based on data from professional race drivers. The herein proposed Driver Identification and Metric Ranking Algorithm (DIMRA) calculates an extensive set of lap-based metrics describing the driving style, mainly depending on the driver actions. To maintain interpretability and reduce complexity, DIMRA reduces the number of metrics using Sequential Backward Selection (SBS) combined with k-medoids clustering. The resulting track-depending subsets facilitate linear separability of driving styles with purities of up to 98% and demonstrate transferability from simulation to real track data. In addition, the previously proposed ProMoD framework is considerably extended and adapted to a complex motorsport driving simulator environment. Besides utilizing a new distance-based ProMP representation and validating driving line samples, the previous behavioral cloning approach is modified to prevent divergence due to the known issue of compounding errors. The resulting driver model can generate complete laps with competitive lap times while maintaining human-like behavior in actions and the utilization of driver assistance systems. Even on mirrored tracks and with considerably reduced tire grip ProMoD can approximately produce expected lap times. The third part of this thesis focuses on providing generalization and learning abilities for the introduced driver model. For this purpose, basic learning procedures and adaptation techniques of race drivers are identified from the literature and extended by an expert interview with a professional race engineer. Based on the hereby generated insights, two methods are developed to further enhance the imitation capabilities of ProMoD: By introducing an approach to estimate target trajectory distributions for unknown tracks based on experiences from other tracks, the driver model achieves track generalization capabilities and is enabled to generate close to competitive lap times on new tracks. Furthermore, an adaptation process is presented that enables ProMoD to adapt its behavior based on experiences from previous laps. By using the professional racing simulator for evaluation, it is shown that the driver model can learn from mistakes from past laps and completes previously unfinished laps with increased performance. In summary, the findings of this thesis contribute to a better understanding of the human driver and pave the way for advanced lap time optimization with consideration of individual driver characteristics. Furthermore, the findings aid enhanced modern vehicle development and potentially future autonomous racing. Besides this race driving setting, such a driver model should be particularly helpful for the development of driver assistance systems in road cars, as it can trigger human-like Traction Control (TC) interventions. Due to its modular architecture, ProMoD may be extended in various ways in future research. Additional adaptation methods may be introduced that further increase the ability of the driver model to learn from experience and modified feature definitions might yield an even more accurate imitation of the human driver. Furthermore, direct encoding of prior knowledge, as the characteristics of the vehicle setup, could further enhance the learning process. In addition, the neural network which models the internal action selection process of the human driver could be directly adapted, for instance by using reinforcement learning techniques. Ultimately, the proposed framework may be applied to comparable, challenging vehicle control tasks like drifting a car, maneuvering drones, or flying planes. |
||||
Alternatives oder übersetztes Abstract: |
|
||||
Status: | Verlagsversion | ||||
URN: | urn:nbn:de:tuda-tuprints-202188 | ||||
Sachgruppe der Dewey Dezimalklassifikatin (DDC): | 000 Allgemeines, Informatik, Informationswissenschaft > 004 Informatik 600 Technik, Medizin, angewandte Wissenschaften > 600 Technik 600 Technik, Medizin, angewandte Wissenschaften > 620 Ingenieurwissenschaften und Maschinenbau |
||||
Fachbereich(e)/-gebiet(e): | 20 Fachbereich Informatik 20 Fachbereich Informatik > Intelligente Autonome Systeme |
||||
Hinterlegungsdatum: | 29 Mär 2022 12:02 | ||||
Letzte Änderung: | 30 Mär 2022 07:38 | ||||
PPN: | |||||
Referenten: | Peters, Prof. Dr. Jan ; Gerdes, Prof. Dr. J. Christian | ||||
Datum der mündlichen Prüfung / Verteidigung / mdl. Prüfung: | 25 Oktober 2021 | ||||
Export: | |||||
Suche nach Titel in: | TUfind oder in Google |
Frage zum Eintrag |
Optionen (nur für Redakteure)
Redaktionelle Details anzeigen |