Moos, Janosch ; Hansel, Kay ; Abdulsamad, Hany ; Stark, Svenja ; Clever, Debora ; Peters, Jan (2022)
Robust Reinforcement Learning: A Review of Foundations and Recent Advances.
In: Machine Learning and Knowledge Extraction, 2022, 4 (1)
doi: 10.26083/tuprints-00021118
Artikel, Zweitveröffentlichung, Verlagsversion
 | Es ist eine neuere Version dieses Eintrags verfügbar. |
Kurzbeschreibung (Abstract)
Reinforcement learning (RL) has become a highly successful framework for learning in Markov decision processes (MDP). Due to the adoption of RL in realistic and complex environments, solution robustness becomes an increasingly important aspect of RL deployment. Nevertheless, current RL algorithms struggle with robustness to uncertainty, disturbances, or structural changes in the environment. We survey the literature on robust approaches to reinforcement learning and categorize these methods in four different ways: (i) Transition robust designs account for uncertainties in the system dynamics by manipulating the transition probabilities between states; (ii) Disturbance robust designs leverage external forces to model uncertainty in the system behavior; (iii) Action robust designs redirect transitions of the system by corrupting an agent’s output; (iv) Observation robust designs exploit or distort the perceived system state of the policy. Each of these robust designs alters a different aspect of the MDP. Additionally, we address the connection of robustness to the risk-based and entropy-regularized RL formulations. The resulting survey covers all fundamental concepts underlying the approaches to robust reinforcement learning and their recent advances.
| Typ des Eintrags: | Artikel |
|---|---|
| Erschienen: | 2022 |
| Autor(en): | Moos, Janosch ; Hansel, Kay ; Abdulsamad, Hany ; Stark, Svenja ; Clever, Debora ; Peters, Jan |
| Art des Eintrags: | Zweitveröffentlichung |
| Titel: | Robust Reinforcement Learning: A Review of Foundations and Recent Advances |
| Sprache: | Englisch |
| Publikationsjahr: | 2022 |
| Publikationsdatum der Erstveröffentlichung: | 2022 |
| Verlag: | MDPI |
| Titel der Zeitschrift, Zeitung oder Schriftenreihe: | Machine Learning and Knowledge Extraction |
| Jahrgang/Volume einer Zeitschrift: | 4 |
| (Heft-)Nummer: | 1 |
| DOI: | 10.26083/tuprints-00021118 |
| URL / URN: | https://tuprints.ulb.tu-darmstadt.de/21118 |
| Zugehörige Links: | |
| Herkunft: | Zweitveröffentlichung DeepGreen |
| Kurzbeschreibung (Abstract): | Reinforcement learning (RL) has become a highly successful framework for learning in Markov decision processes (MDP). Due to the adoption of RL in realistic and complex environments, solution robustness becomes an increasingly important aspect of RL deployment. Nevertheless, current RL algorithms struggle with robustness to uncertainty, disturbances, or structural changes in the environment. We survey the literature on robust approaches to reinforcement learning and categorize these methods in four different ways: (i) Transition robust designs account for uncertainties in the system dynamics by manipulating the transition probabilities between states; (ii) Disturbance robust designs leverage external forces to model uncertainty in the system behavior; (iii) Action robust designs redirect transitions of the system by corrupting an agent’s output; (iv) Observation robust designs exploit or distort the perceived system state of the policy. Each of these robust designs alters a different aspect of the MDP. Additionally, we address the connection of robustness to the risk-based and entropy-regularized RL formulations. The resulting survey covers all fundamental concepts underlying the approaches to robust reinforcement learning and their recent advances. |
| Freie Schlagworte: | reinforcement learning, robustness, min-max optimization |
| Status: | Verlagsversion |
| URN: | urn:nbn:de:tuda-tuprints-211188 |
| Sachgruppe der Dewey Dezimalklassifikatin (DDC): | 000 Allgemeines, Informatik, Informationswissenschaft > 004 Informatik 600 Technik, Medizin, angewandte Wissenschaften > 620 Ingenieurwissenschaften und Maschinenbau |
| Fachbereich(e)/-gebiet(e): | 16 Fachbereich Maschinenbau 16 Fachbereich Maschinenbau > Institut für Mechatronische Systeme im Maschinenbau (IMS) 20 Fachbereich Informatik 20 Fachbereich Informatik > Intelligente Autonome Systeme |
| Hinterlegungsdatum: | 11 Apr 2022 11:34 |
| Letzte Änderung: | 12 Apr 2022 05:08 |
| PPN: | |
| Export: | |
| Suche nach Titel in: | TUfind oder in Google |
Verfügbare Versionen dieses Eintrags
- Robust Reinforcement Learning: A Review of Foundations and Recent Advances. (deposited 11 Apr 2022 11:34) [Gegenwärtig angezeigt]
 |
Frage zum Eintrag |
Optionen (nur für Redakteure)
 |
Redaktionelle Details anzeigen |
Drucken
Impressum