Cui, Kai ; Koeppl, Heinz (2022)
Approximately Solving Mean Field Games via Entropy-Regularized Deep Reinforcement Learning.
24th International Conference on Artificial Intelligence and Statistics (AISTATS) 2021. Virtual (13.-15.04.2021)
doi: 10.26083/tuprints-00021511
Konferenzveröffentlichung, Zweitveröffentlichung, Verlagsversion
Kurzbeschreibung (Abstract)
The recent mean field game (MFG) formalism facilitates otherwise intractable computation of approximate Nash equilibria in many-agent settings. In this paper, we consider discrete-time finite MFGs subject to finite-horizon objectives. We show that all discrete-time finite MFGs with non-constant fixed point operators fail to be contractive as typically assumed in existing MFG literature, barring convergence via fixed point iteration. Instead, we incorporate entropy-regularization and Boltzmann policies into the fixed point iteration. As a result, we obtain provable convergence to approximate fixed points where existing methods fail, and reach the original goal of approximate Nash equilibria. All proposed methods are evaluated with respect to their exploitability, on both instructive examples with tractable exact solutions and high-dimensional problems where exact methods become intractable. In high-dimensional scenarios, we apply established deep reinforcement learning methods and empirically combine fictitious play with our approximations.
| Typ des Eintrags: | Konferenzveröffentlichung |
|---|---|
| Erschienen: | 2022 |
| Autor(en): | Cui, Kai ; Koeppl, Heinz |
| Art des Eintrags: | Zweitveröffentlichung |
| Titel: | Approximately Solving Mean Field Games via Entropy-Regularized Deep Reinforcement Learning |
| Sprache: | Englisch |
| Publikationsjahr: | 2022 |
| Ort: | Darmstadt |
| Verlag: | PMLR |
| Buchtitel: | Proceedings of The 24th International Conference on Artificial Intelligence and Statistics |
| Reihe: | Proceedings of Machine Learning Research |
| Band einer Reihe: | 130 |
| Veranstaltungstitel: | 24th International Conference on Artificial Intelligence and Statistics (AISTATS) 2021 |
| Veranstaltungsort: | Virtual |
| Veranstaltungsdatum: | 13.-15.04.2021 |
| DOI: | 10.26083/tuprints-00021511 |
| URL / URN: | https://tuprints.ulb.tu-darmstadt.de/21511 |
| Zugehörige Links: | |
| Herkunft: | Zweitveröffentlichungsservice |
| Kurzbeschreibung (Abstract): | The recent mean field game (MFG) formalism facilitates otherwise intractable computation of approximate Nash equilibria in many-agent settings. In this paper, we consider discrete-time finite MFGs subject to finite-horizon objectives. We show that all discrete-time finite MFGs with non-constant fixed point operators fail to be contractive as typically assumed in existing MFG literature, barring convergence via fixed point iteration. Instead, we incorporate entropy-regularization and Boltzmann policies into the fixed point iteration. As a result, we obtain provable convergence to approximate fixed points where existing methods fail, and reach the original goal of approximate Nash equilibria. All proposed methods are evaluated with respect to their exploitability, on both instructive examples with tractable exact solutions and high-dimensional problems where exact methods become intractable. In high-dimensional scenarios, we apply established deep reinforcement learning methods and empirically combine fictitious play with our approximations. |
| Status: | Verlagsversion |
| URN: | urn:nbn:de:tuda-tuprints-215111 |
| Sachgruppe der Dewey Dezimalklassifikatin (DDC): | 000 Allgemeines, Informatik, Informationswissenschaft > 004 Informatik 500 Naturwissenschaften und Mathematik > 510 Mathematik 600 Technik, Medizin, angewandte Wissenschaften > 620 Ingenieurwissenschaften und Maschinenbau |
| Fachbereich(e)/-gebiet(e): | 18 Fachbereich Elektrotechnik und Informationstechnik 18 Fachbereich Elektrotechnik und Informationstechnik > Institut für Nachrichtentechnik > Bioinspirierte Kommunikationssysteme 18 Fachbereich Elektrotechnik und Informationstechnik > Institut für Nachrichtentechnik 18 Fachbereich Elektrotechnik und Informationstechnik > Self-Organizing Systems Lab |
| TU-Projekte: | HMWK|III L6-519/03/05.001-(0016)|emergenCity TP Bock |
| Hinterlegungsdatum: | 20 Jul 2022 13:34 |
| Letzte Änderung: | 26 Jul 2022 09:49 |
| PPN: | |
| Zugehörige Links: | |
| Export: | |
| Suche nach Titel in: | TUfind oder in Google |
 |
Frage zum Eintrag |
Optionen (nur für Redakteure)
 |
Redaktionelle Details anzeigen |
Drucken
Impressum