Cui, Kai and Koeppl, Heinz (2021):
Approximately Solving Mean Field Games via Entropy-Regularized Deep Reinforcement Learning.
24th International Conference on Artificial Intelligence and Statistics, Virtual Conference, 13.-15.04.2021, [Conference or Workshop Item]
Abstract
The recent mean field game (MFG) formalism facilitates otherwise intractable computation of approximate Nash equilibria in many-agent settings. In this paper, we consider discrete-time finite MFGs subject to finite-horizon objectives. We show that all discrete-time finite MFGs with non-constant fixed point operators fail to be contractive as typically assumed in existing MFG literature, barring convergence via fixed point iteration. Instead, we incorporate entropy-regularization and Boltzmann policies into the fixed point iteration. As a result, we obtain provable convergence to approximate fixed points where existing methods fail, and reach the original goal of approximate Nash equilibria. All proposed methods are evaluated with respect to their exploitability, on both instructive examples with tractable exact solutions and high-dimensional problems where exact methods become intractable. In high-dimensional scenarios, we apply established deep reinforcement learning methods and empirically combine fictitious play with our approximations.
| Item Type: | Conference or Workshop Item |
|---|---|
| Erschienen: | 2021 |
| Creators: | Cui, Kai and Koeppl, Heinz |
| Title: | Approximately Solving Mean Field Games via Entropy-Regularized Deep Reinforcement Learning |
| Language: | English |
| Abstract: | The recent mean field game (MFG) formalism facilitates otherwise intractable computation of approximate Nash equilibria in many-agent settings. In this paper, we consider discrete-time finite MFGs subject to finite-horizon objectives. We show that all discrete-time finite MFGs with non-constant fixed point operators fail to be contractive as typically assumed in existing MFG literature, barring convergence via fixed point iteration. Instead, we incorporate entropy-regularization and Boltzmann policies into the fixed point iteration. As a result, we obtain provable convergence to approximate fixed points where existing methods fail, and reach the original goal of approximate Nash equilibria. All proposed methods are evaluated with respect to their exploitability, on both instructive examples with tractable exact solutions and high-dimensional problems where exact methods become intractable. In high-dimensional scenarios, we apply established deep reinforcement learning methods and empirically combine fictitious play with our approximations. |
| Uncontrolled Keywords: | emergenCITY_KOM |
| Divisions: | 18 Department of Electrical Engineering and Information Technology 18 Department of Electrical Engineering and Information Technology > Institute for Telecommunications > Bioinspired Communication Systems 18 Department of Electrical Engineering and Information Technology > Institute for Telecommunications LOEWE LOEWE > LOEWE-Zentren LOEWE > LOEWE-Zentren > emergenCITY |
| TU-Projects: | HMWK|III L6-519/03/05.001-(0016)|emergenCity TP Bock |
| Event Title: | 24th International Conference on Artificial Intelligence and Statistics |
| Event Location: | Virtual Conference |
| Event Dates: | 13.-15.04.2021 |
| Date Deposited: | 22 Feb 2021 07:28 |
| Corresponding Links: | |
| Export: | |
| Suche nach Titel in: | TUfind oder in Google |
 |
Send an inquiry |
Options (only for editors)
 |
Show editorial Details |
Drucken
Impressum